From 1bf64b310274a5907c4c38299cec04e9c998da1c Mon Sep 17 00:00:00 2001 From: Oleg Ivaniv Date: Thu, 25 Apr 2024 13:58:57 +0200 Subject: [PATCH] Update Ollama chat + lm options docs Signed-off-by: Oleg Ivaniv --- .../n8n-nodes-langchain.lmchatollama.md | 26 +++++++++++++----- .../sub-nodes/n8n-nodes-langchain.lmollama.md | 27 +++++++++++++------ 2 files changed, 38 insertions(+), 15 deletions(-) diff --git a/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama.md b/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama.md index 4a5c15b60..782116dfb 100644 --- a/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama.md +++ b/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmchatollama.md @@ -21,17 +21,29 @@ For usage examples and templates to help you get started, refer to n8n's [Ollama ## Node parameters -* **Model**: the model that generates the completion. Choose from: - * Llama2 - * Llama2 13B - * Llama2 70B - * Llama2 Uncensored +* **Model**: the model that generates the completion. The models options are fetched from the Ollama API running on the provided credential URL. ## Node options * **Sampling Temperature**: controls the randomness of the sampling process. A higher temperature creates more diverse sampling, but increases the risk of hallucinations. -* **Top K**: the number of token choices the model uses to generate the next token. -* **Top P**: use a lower value to ignore less probable options. +* **Top K**: limits the number of highest probability vocabulary tokens to consider at each step. A higher value increases diversity but may reduce coherence. Set to -1 to disable. Valid range is between -1 and 100. +* **Top P**: chooses from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Helps generate more human-like text by reducing repetitions. Valid range is between 0 and 1. +* **Frequency Penalty**: adjusts the penalty for tokens that have already appeared in the generated text. Higher values discourage repetition. Must be a non-negative number. +* **Keep Alive**: specifies the duration to keep the loaded model in memory after use. Useful for frequently used models. Format: 1h30m (1 hour 30 minutes). +* **Low VRAM Mode**: whether to activate low VRAM mode, which reduces memory usage at the cost of slower generation speed. Useful for GPUs with limited memory. +* **Main GPU ID**: specifies the ID of the GPU to use for the main computation. Only change this if you have multiple GPUs. +* **Context Batch Size**: sets the batch size for prompt processing. Larger batch sizes may improve generation speed but increase memory usage. +* **Context Length**: the maximum number of tokens to use as context for generating the next token. Smaller values reduce memory usage, while larger values provide more context to the model. +* **Number of GPUs**: specifies the number of GPUs to use for parallel processing. Set to -1 for auto-detection. +* **Max Tokens to Generate**: the maximum number of tokens to generate. Set to -1 for no limit. Be cautious when setting this to a large value, as it can lead to very long outputs. +* **Number of CPU Threads**: specifies the number of CPU threads to use for processing. Set to 0 for auto-detection. +* **Penalize Newlines**: whether the model will be less likely to generate newline characters, encouraging longer continuous sequences of text. +* **Presence Penalty**: adjusts the penalty for tokens based on their presence in the generated text so far. Positive values penalize tokens that have already appeared, encouraging diversity. +* **Repetition Penalty**: adjusts the penalty factor for repeated tokens. Higher values more strongly discourage repetition. Set to 1.0 to disable repetition penalty. +* **Use Memory Locking**: whether to lock the model in memory to prevent swapping. This can improve performance but requires sufficient available memory. +* **Use Memory Mapping**: whether to use memory mapping for loading the model. This can reduce memory usage but may impact performance. Recommended to keep enabled. +* **Load Vocabulary Only**: whether to only load the model vocabulary without the weights. Useful for quickly testing tokenization. +* **Output Format**: specifies the format of the API response. Choose between 'Default' and 'JSON'. ## Related resources diff --git a/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmollama.md b/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmollama.md index b56c1e978..4555b0a96 100644 --- a/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmollama.md +++ b/docs/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.lmollama.md @@ -20,18 +20,29 @@ For usage examples and templates to help you get started, refer to n8n's [Ollama ## Node parameters -* **Model**: the model that generates the completion. Choose from: - * Llama2 - * Llama2 13B - * Llama2 70B - * Llama2 Uncensored +* **Model**: the model that generates the completion. The models options are fetched from the Ollama API running on the provided credential URL. ## Node options * **Sampling Temperature**: controls the randomness of the sampling process. A higher temperature creates more diverse sampling, but increases the risk of hallucinations. -* **Top K**: the number of token choices the model uses to generate the next token. -* **Top P**: use a lower value to ignore less probable options. - +* **Top K**: limits the number of highest probability vocabulary tokens to consider at each step. A higher value increases diversity but may reduce coherence. Set to -1 to disable. Valid range is between -1 and 100. +* **Top P**: chooses from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Helps generate more human-like text by reducing repetitions. Valid range is between 0 and 1. +* **Frequency Penalty**: adjusts the penalty for tokens that have already appeared in the generated text. Higher values discourage repetition. Must be a non-negative number. +* **Keep Alive**: specifies the duration to keep the loaded model in memory after use. Useful for frequently used models. Format: 1h30m (1 hour 30 minutes). +* **Low VRAM Mode**: whether to activate low VRAM mode, which reduces memory usage at the cost of slower generation speed. Useful for GPUs with limited memory. +* **Main GPU ID**: specifies the ID of the GPU to use for the main computation. Only change this if you have multiple GPUs. +* **Context Batch Size**: sets the batch size for prompt processing. Larger batch sizes may improve generation speed but increase memory usage. +* **Context Length**: the maximum number of tokens to use as context for generating the next token. Smaller values reduce memory usage, while larger values provide more context to the model. +* **Number of GPUs**: specifies the number of GPUs to use for parallel processing. Set to -1 for auto-detection. +* **Max Tokens to Generate**: the maximum number of tokens to generate. Set to -1 for no limit. Be cautious when setting this to a large value, as it can lead to very long outputs. +* **Number of CPU Threads**: specifies the number of CPU threads to use for processing. Set to 0 for auto-detection. +* **Penalize Newlines**: whether the model will be less likely to generate newline characters, encouraging longer continuous sequences of text. +* **Presence Penalty**: adjusts the penalty for tokens based on their presence in the generated text so far. Positive values penalize tokens that have already appeared, encouraging diversity. +* **Repetition Penalty**: adjusts the penalty factor for repeated tokens. Higher values more strongly discourage repetition. Set to 1.0 to disable repetition penalty. +* **Use Memory Locking**: whether to lock the model in memory to prevent swapping. This can improve performance but requires sufficient available memory. +* **Use Memory Mapping**: whether to use memory mapping for loading the model. This can reduce memory usage but may impact performance. Recommended to keep enabled. +* **Load Vocabulary Only**: whether to only load the model vocabulary without the weights. Useful for quickly testing tokenization. +* **Output Format**: specifies the format of the API response. Choose between 'Default' and 'JSON'. ## Related resources