mirrors/n8n-docs
2
0
Fork 0
mirror of https://github.com/n8n-io/n8n-docs.git synced 2025-11-20 17:48:34 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Update Ollama chat + lm options docs

Browse Source 
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
This commit is contained in:
Oleg Ivaniv 2024-04-25 13:58:57 +02:00
parent eef52f34d3
commit 1bf64b3102
No known key found for this signature in database
2 changed files with 38 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama.md
View File

@ -21,17 +21,29 @@ For usage examples and templates to help you get started, refer to n8n's [Ollama
## Node parameters
* **Model**: the model that generates the completion. Choose from:
* Llama2
* Llama2 13B
* Llama2 70B
* Llama2 Uncensored
* **Model**: the model that generates the completion. The models options are fetched from the Ollama API running on the provided credential URL.
## Node options
* **Sampling Temperature**: controls the randomness of the sampling process. A higher temperature creates more diverse sampling, but increases the risk of hallucinations.
* **Top K**: the number of token choices the model uses to generate the next token.
* **Top P**: use a lower value to ignore less probable options.
* **Top K**: limits the number of highest probability vocabulary tokens to consider at each step. A higher value increases diversity but may reduce coherence. Set to -1 to disable. Valid range is between -1 and 100.
* **Top P**: chooses from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Helps generate more human-like text by reducing repetitions. Valid range is between 0 and 1.
* **Frequency Penalty**: adjusts the penalty for tokens that have already appeared in the generated text. Higher values discourage repetition. Must be a non-negative number.
* **Keep Alive**: specifies the duration to keep the loaded model in memory after use. Useful for frequently used models. Format: 1h30m (1 hour 30 minutes).
* **Low VRAM Mode**: whether to activate low VRAM mode, which reduces memory usage at the cost of slower generation speed. Useful for GPUs with limited memory.
* **Main GPU ID**: specifies the ID of the GPU to use for the main computation. Only change this if you have multiple GPUs.
* **Context Batch Size**: sets the batch size for prompt processing. Larger batch sizes may improve generation speed but increase memory usage.
* **Context Length**: the maximum number of tokens to use as context for generating the next token. Smaller values reduce memory usage, while larger values provide more context to the model.
* **Number of GPUs**: specifies the number of GPUs to use for parallel processing. Set to -1 for auto-detection.
* **Max Tokens to Generate**: the maximum number of tokens to generate. Set to -1 for no limit. Be cautious when setting this to a large value, as it can lead to very long outputs.
* **Number of CPU Threads**: specifies the number of CPU threads to use for processing. Set to 0 for auto-detection.
* **Penalize Newlines**: whether the model will be less likely to generate newline characters, encouraging longer continuous sequences of text.
* **Presence Penalty**: adjusts the penalty for tokens based on their presence in the generated text so far. Positive values penalize tokens that have already appeared, encouraging diversity.
* **Repetition Penalty**: adjusts the penalty factor for repeated tokens. Higher values more strongly discourage repetition. Set to 1.0 to disable repetition penalty.
* **Use Memory Locking**: whether to lock the model in memory to prevent swapping. This can improve performance but requires sufficient available memory.
* **Use Memory Mapping**: whether to use memory mapping for loading the model. This can reduce memory usage but may impact performance. Recommended to keep enabled.
* **Load Vocabulary Only**: whether to only load the model vocabulary without the weights. Useful for quickly testing tokenization.
* **Output Format**: specifies the format of the API response. Choose between 'Default' and 'JSON'.
## Related resources

27
docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmollama.md
View File

@ -20,18 +20,29 @@ For usage examples and templates to help you get started, refer to n8n's [Ollama
## Node parameters
* **Model**: the model that generates the completion. Choose from:
* Llama2
* Llama2 13B
* Llama2 70B
* Llama2 Uncensored
* **Model**: the model that generates the completion. The models options are fetched from the Ollama API running on the provided credential URL.
## Node options
* **Sampling Temperature**: controls the randomness of the sampling process. A higher temperature creates more diverse sampling, but increases the risk of hallucinations.
* **Top K**: the number of token choices the model uses to generate the next token.
* **Top P**: use a lower value to ignore less probable options.
* **Top K**: limits the number of highest probability vocabulary tokens to consider at each step. A higher value increases diversity but may reduce coherence. Set to -1 to disable. Valid range is between -1 and 100.
* **Top P**: chooses from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Helps generate more human-like text by reducing repetitions. Valid range is between 0 and 1.
* **Frequency Penalty**: adjusts the penalty for tokens that have already appeared in the generated text. Higher values discourage repetition. Must be a non-negative number.
* **Keep Alive**: specifies the duration to keep the loaded model in memory after use. Useful for frequently used models. Format: 1h30m (1 hour 30 minutes).
* **Low VRAM Mode**: whether to activate low VRAM mode, which reduces memory usage at the cost of slower generation speed. Useful for GPUs with limited memory.
* **Main GPU ID**: specifies the ID of the GPU to use for the main computation. Only change this if you have multiple GPUs.
* **Context Batch Size**: sets the batch size for prompt processing. Larger batch sizes may improve generation speed but increase memory usage.
* **Context Length**: the maximum number of tokens to use as context for generating the next token. Smaller values reduce memory usage, while larger values provide more context to the model.
* **Number of GPUs**: specifies the number of GPUs to use for parallel processing. Set to -1 for auto-detection.
* **Max Tokens to Generate**: the maximum number of tokens to generate. Set to -1 for no limit. Be cautious when setting this to a large value, as it can lead to very long outputs.
* **Number of CPU Threads**: specifies the number of CPU threads to use for processing. Set to 0 for auto-detection.
* **Penalize Newlines**: whether the model will be less likely to generate newline characters, encouraging longer continuous sequences of text.
* **Presence Penalty**: adjusts the penalty for tokens based on their presence in the generated text so far. Positive values penalize tokens that have already appeared, encouraging diversity.
* **Repetition Penalty**: adjusts the penalty factor for repeated tokens. Higher values more strongly discourage repetition. Set to 1.0 to disable repetition penalty.
* **Use Memory Locking**: whether to lock the model in memory to prevent swapping. This can improve performance but requires sufficient available memory.
* **Use Memory Mapping**: whether to use memory mapping for loading the model. This can reduce memory usage but may impact performance. Recommended to keep enabled.
* **Load Vocabulary Only**: whether to only load the model vocabulary without the weights. Useful for quickly testing tokenization.
* **Output Format**: specifies the format of the API response. Choose between 'Default' and 'JSON'.
## Related resources