fix: allow configuring Ollama context size via OLLAMA_NUM_CTX env var

[octo-patch]

Fixes #981

Problem

The num_ctx parameter in the Ollama LLM provider was hardcoded to 32000 across all four generation methods (generateText, streamText, generateObject, streamObject). This caused Ollama to unload and reload models with a 32K context window, even when users had configured a larger context size in their Ollama model settings. Using the OpenAI-compatible provider with the same model would respect the configured context size, but the native Ollama provider would not.

Solution

Read the context size from the OLLAMA_NUM_CTX environment variable, falling back to 32000 if not set. This maintains backward compatibility while allowing users to configure the context size to match their model's capabilities:

# Example: set 128K context for all Ollama models
OLLAMA_NUM_CTX=131072

The fix also adds num_ctx to generateObject and streamObject methods, which previously omitted it entirely (inconsistent with the other methods).

Testing

• Verified the environment variable is read at module load time
• Default behavior (32000) is preserved when OLLAMA_NUM_CTX is not set
• Setting OLLAMA_NUM_CTX=131072 will pass 131072 as the context size to all Ollama calls

---

Summary by cubic

Make Ollama context size configurable via OLLAMA_NUM_CTX instead of hardcoded 32K, avoiding unnecessary model reloads and matching each model’s configured window. Also adds num_ctx to object generation methods for consistency.

• Bug Fixes
  • Read OLLAMA_NUM_CTX at module load; fallback to 32000 if unset.
  • Apply num_ctx to generateText, streamText, generateObject, and streamObject.
  • Preserves previous default behavior; set OLLAMA_NUM_CTX to match your model’s context size.

^{Written for commit a520ace. Summary will update on new commits.}

[cubic-dev-ai]

1 issue found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/lib/models/providers/ollama/ollamaLLM.ts">

<violation number="1" location="src/lib/models/providers/ollama/ollamaLLM.ts:24">
P2: `OLLAMA_NUM_CTX` parsing lacks validation; non-numeric truthy values produce `NaN`, which is then sent as `num_ctx` for all Ollama calls.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: OLLAMA_NUM_CTX parsing lacks validation; non-numeric truthy values produce NaN, which is then sent as num_ctx for all Ollama calls.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/lib/models/providers/ollama/ollamaLLM.ts, line 24:

<comment>`OLLAMA_NUM_CTX` parsing lacks validation; non-numeric truthy values produce `NaN`, which is then sent as `num_ctx` for all Ollama calls.</comment>

<file context>
@@ -19,6 +19,11 @@ type OllamaConfig = {
 
+const DEFAULT_OLLAMA_NUM_CTX = 32000;
+const numCtx = process.env.OLLAMA_NUM_CTX
+  ? parseInt(process.env.OLLAMA_NUM_CTX, 10)
+  : DEFAULT_OLLAMA_NUM_CTX;
+
</file context>