fix: respect Ollama Modelfile num_ctx configuration #7798

roomote · 2025-09-08T21:58:09Z

Description

This PR fixes an issue where Roo was overriding Ollama's Modelfile setting by forcing from the inferred . This was causing memory allocation problems on budget GPUs when users had specifically configured smaller context windows in their Modelfiles.

Problem

Roo was sending equal to its inferred on every chat request
This overrode the user's Modelfile configuration
Could allocate much larger context windows than intended (e.g., 128k/200k instead of 4k)
Caused OOM errors on budget GPUs

Solution

Removed automatic from default chat options in
Added optional parameter to for explicit overrides
Updated both and to only include when explicitly set
Now respects Ollama's Modelfile configuration by default

Changes

src/shared/api.ts: Added optional field to interface
src/api/providers/native-ollama.ts: Modified to conditionally include only when explicitly set
src/api/providers/tests/native-ollama.spec.ts: Added tests to verify is not sent by default

Testing

✅ All existing tests pass
✅ Added 4 new test cases to verify:
- is not included by default in streaming mode
- is included when explicitly set in streaming mode
- is not included by default in non-streaming mode
- is included when explicitly set in non-streaming mode

Impact

Non-breaking change: Existing behavior is preserved for users who don't use Ollama
Backward compatible: Users can still override if needed via the new option
Respects user configuration: Ollama will now use the Modelfile's or server-side defaults

Fixes #7797

Important

Fixes issue with Ollama's num_ctx configuration by respecting Modelfile settings by default and allowing explicit overrides.

Behavior:
- Removes automatic num_ctx from default chat options in native-ollama.ts.
- Adds optional ollamaNumCtx parameter to ApiHandlerOptions in api.ts for explicit overrides.
- Updates native-ollama.ts to include num_ctx only when explicitly set.
Testing:
- Adds tests in native-ollama.spec.ts to verify num_ctx behavior in both streaming and non-streaming modes.
UI:
- Adds ollamaNumCtx input field in Ollama.tsx for user configuration.
Localization:
- Updates localization files to include numCtx and numCtxHelp descriptions in multiple languages.

^{This description was created by}^{for 0c436e0. You can customize this summary. It will automatically update as commits are pushed.}

roomote

I reviewed my own code and found issues I should have caught before pushing. Classic.

Critical Issues

Missing UI support - The new ollamaNumCtx field has no UI component in the settings panel, so users cannot configure this value through the interface.
Missing type definition - The ollamaNumCtx field should be added to the ProviderSettings schema in packages/types/src/provider-settings.ts for proper type safety.

Suggestions

Consider using proper TypeScript types instead of any for the chat options
Add validation for the ollamaNumCtx value to ensure it's within reasonable bounds
The documentation could be more explicit about when to use this override

roomote · 2025-09-08T22:05:11Z

src/api/providers/native-ollama.ts


 		try {
+			// Build options object conditionally
+			const chatOptions: any = {


Using any type here bypasses TypeScript's type checking. Consider defining a proper interface:

const chatOptions: { temperature: number; num_ctx?: number } = { temperature: this.options.modelTemperature ?? (useR1Format ? DEEP_SEEK_DEFAULT_TEMPERATURE : 0), }

roomote · 2025-09-08T22:05:11Z

src/api/providers/native-ollama.ts

 			const useR1Format = modelId.toLowerCase().includes("deepseek-r1")

+			// Build options object conditionally
+			const chatOptions: any = {


Same type safety issue here. Should use a proper type instead of any.

roomote · 2025-09-08T22:05:11Z

src/shared/api.ts

+	 * When set, this value will be used in Ollama chat requests.
+	 * When undefined, Ollama will use the model's default num_ctx from the Modelfile.
+	 */
+	ollamaNumCtx?: number


Is this intentional that we're not adding this field to the ProviderSettings type in packages/types? Without that, the field won't be properly validated when settings are saved/loaded.

Should these be added to the ProviderSettings?

roomote · 2025-09-08T22:05:11Z

src/api/providers/__tests__/native-ollama.spec.ts

+				apiModelId: "llama2",
+				ollamaModelId: "llama2",
+				ollamaBaseUrl: "http://localhost:11434",
+				ollamaNumCtx: 8192, // Explicitly set num_ctx


Consider adding tests for edge cases like negative numbers or extremely large values for ollamaNumCtx. Validation tests would help ensure the field only accepts reasonable values.

daniel-lxs · 2025-09-09T18:01:51Z

@hannesrudolph it seems that there's no way to set the new setting since there's no UI changes

hannesrudolph · 2025-09-09T18:06:56Z

@roomote-agent lets add a setting for setting to set the CTX or whatever?

roomote · 2025-09-09T18:08:32Z

Hi @hannesrudolph! I see the request - I'll add a UI setting for configuring the Ollama context window (num_ctx). This will allow users to set this value through the settings panel. Working on it now!

roomote · 2025-09-09T18:24:13Z

Hi @hannesrudolph! I've successfully added the UI setting for Ollama's num_ctx configuration. Here's what was implemented:

Changes Made:

✅ Added ollamaNumCtx field to the provider settings schema in packages/types/src/provider-settings.ts
✅ Added a number input field in the Ollama settings UI (webview-ui/src/components/settings/providers/Ollama.tsx)
✅ Added English translation keys for the new setting
✅ Validated that minimum value is 128 tokens to prevent invalid configurations

How it works:

Users can now override the default context window size through the settings UI
When left empty, Ollama will use the model's Modelfile configuration (default behavior)
When set, the value will be passed as num_ctx to Ollama API calls
The backend implementation (already in the PR) will use this value when making requests

CI Status:

Most checks are passing. The translation check is failing because translations for other languages need to be added - this is typically handled by the translation team or can be added in a follow-up.

The feature is ready to use! Users can now configure the context window size for their Ollama models directly from the settings panel.

hannesrudolph · 2025-09-09T19:07:55Z

@hannesrudolph it seems that there's no way to set the new setting since there's no UI changes

Tested with and without this fix. So not only does the config now respect what the user has set in ollama, it can be overidden with a new box that shows up in the config.

Copilot

Pull Request Overview

This PR fixes an issue where Roo was forcefully overriding Ollama's Modelfile num_ctx configuration, causing memory allocation problems on budget GPUs. Previously, Roo automatically sent num_ctx equal to its inferred context window on every chat request, ignoring users' intentional Modelfile configurations.

Removed automatic num_ctx parameter from default Ollama chat options
Added optional ollamaNumCtx parameter for explicit context window overrides
Updated both streaming and non-streaming Ollama methods to conditionally include num_ctx

Reviewed Changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/shared/api.ts	Added `ollamaNumCtx` optional field to `ApiHandlerOptions` interface
src/api/providers/native-ollama.ts	Modified to conditionally include `num_ctx` only when explicitly set via `ollamaNumCtx`
src/api/providers/tests/native-ollama.spec.ts	Added comprehensive tests verifying conditional `num_ctx` behavior
packages/types/src/provider-settings.ts	Added `ollamaNumCtx` field to Ollama schema with validation
webview-ui/src/components/settings/providers/Ollama.tsx	Implemented UI input field for configuring context window size
webview-ui/src/i18n/locales/*/settings.json	Added localized strings for new context window configuration options

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-09-09T19:08:28Z

webview-ui/src/i18n/locales/it/settings.json

+			"numCtxHelp": "Sovrascrive la dimensione predefinita della finestra di contesto del modello. Lasciare vuoto per utilizzare la configurazione del Modelfile del modello. Il valore minimo è 128.",
 			"description": "Ollama ti permette di eseguire modelli localmente sul tuo computer. Per iniziare, consulta la guida rapida.",
-			"warning": "Nota: Roo Code utilizza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."
+			"warning": "Nota: Roo Code utiliza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."


The Italian text uses 'utiliza' (Spanish) instead of the correct Italian 'utilizza'. This should be 'utilizza' to match proper Italian grammar.

Suggested change

"warning": "Nota: Roo Code utiliza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."

"warning": "Nota: Roo Code utilizza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."

ellipsis-dev · 2025-09-09T19:10:33Z

webview-ui/src/i18n/locales/it/settings.json

+			"numCtxHelp": "Sovrascrive la dimensione predefinita della finestra di contesto del modello. Lasciare vuoto per utilizzare la configurazione del Modelfile del modello. Il valore minimo è 128.",
 			"description": "Ollama ti permette di eseguire modelli localmente sul tuo computer. Per iniziare, consulta la guida rapida.",
-			"warning": "Nota: Roo Code utilizza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."
+			"warning": "Nota: Roo Code utiliza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."


There appears to be a typographical error: "utiliza" should likely be "utilizza" (in Italian).

Suggested change

"warning": "Nota: Roo Code utiliza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."

"warning": "Nota: Roo Code utilizza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."

roomote bot requested review from cte, jr and mrubens as code owners September 8, 2025 21:58

github-project-automation bot added this to Roo Code Roadmap and Roo Code Roadmap Sep 8, 2025

github-project-automation bot moved this to New in Roo Code Roadmap Sep 8, 2025

github-project-automation bot moved this to Triage in Roo Code Roadmap Sep 8, 2025

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Sep 8, 2025

hannesrudolph added the Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. label Sep 8, 2025

roomote bot commented Sep 8, 2025

View reviewed changes

roomote bot mentioned this pull request Sep 8, 2025

Ollama: Roo overrides Modelfile num_ctx by forcing options.num_ctx from modelInfo.contextWindow #7797

Closed

hannesrudolph moved this from Triage to PR [Needs Prelim Review] in Roo Code Roadmap Sep 8, 2025

hannesrudolph added PR - Needs Preliminary Review and removed Issue/PR - Triage New issue. Needs quick review to confirm validity and assign labels. labels Sep 8, 2025

daniel-lxs assigned hannesrudolph Sep 9, 2025

daniel-lxs marked this pull request as draft September 9, 2025 18:01

daniel-lxs moved this from PR [Needs Prelim Review] to PR [Draft / In Progress] in Roo Code Roadmap Sep 9, 2025

hannesrudolph added PR - Draft / In Progress and removed PR - Needs Preliminary Review labels Sep 9, 2025

hannesrudolph moved this from PR [Draft / In Progress] to PR [Needs Prelim Review] in Roo Code Roadmap Sep 9, 2025

hannesrudolph marked this pull request as ready for review September 9, 2025 19:08

Copilot AI review requested due to automatic review settings September 9, 2025 19:08

Copilot AI reviewed Sep 9, 2025

View reviewed changes

ellipsis-dev bot reviewed Sep 9, 2025

View reviewed changes

	"warning": "Nota: Roo Code utiliza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."
	"warning": "Nota: Roo Code utilizza prompt complessi e funziona meglio con i modelli Claude. I modelli con capacità inferiori potrebbero non funzionare come previsto."

fix: respect Ollama Modelfile num_ctx configuration #7798

fix: respect Ollama Modelfile num_ctx configuration #7798

Uh oh!

Conversation

roomote bot commented Sep 8, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Problem

Solution

Changes

Testing

Impact

Uh oh!

roomote bot left a comment

Choose a reason for hiding this comment

Critical Issues

Suggestions

Uh oh!

roomote bot Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

mrubens Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

roomote bot Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

daniel-lxs commented Sep 9, 2025

Uh oh!

hannesrudolph commented Sep 9, 2025

Uh oh!

roomote bot commented Sep 9, 2025

Uh oh!

roomote bot commented Sep 9, 2025

Changes Made:

How it works:

CI Status:

Uh oh!

hannesrudolph commented Sep 9, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot Sep 9, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

roomote bot commented Sep 8, 2025 •

edited by ellipsis-dev bot

Loading