fix: improve GLM model detection patterns for LM Studio and OpenAI-compatible endpoints #11082
+225
−5
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…mpatible endpoints Addresses Issue #11071 where GLM4.5 models via LM Studio get stuck repeating file reads due to undetected model names. Changes: - Add isGlmModel() utility that uses a flexible regex pattern to detect GLM models anywhere in the model ID (case-insensitive) - Handle model name formats like: - mlx-community/GLM-4.5-4bit - GLM-4.5-UD-Q8_K_XL-00001-of-00008.gguf - glm-4.5 (standard format) - chatglm variants - Update LmStudioHandler to apply GLM-specific optimizations when detected - Update BaseOpenAiCompatibleProvider similarly - Add comprehensive tests for GLM model detection When GLM model is detected: - mergeToolResultText: true (prevents conversation flow disruption) - parallel_tool_calls: false (GLM may not support this parameter)
Contributor
Author
Review complete. No issues found. The implementation is clean and follows established patterns in the codebase. The GLM detection uses the existing Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR attempts to address Issue #11071 where GLM4.5 models via LM Studio or OpenAI-compatible endpoints get stuck repeating file reads.
Problem
The user reported that model names like:
mlx-community/GLM-4.5-4bitGLM-4.5-UD-Q8_K_XL-00001-of-00008.ggufglm-4.5were not being detected as GLM models, causing the optimizations for GLM models to not be applied.
Solution
Created a flexible
isGlmModel()utility function that uses a simple regex pattern (/glm/i) to detect GLM anywhere in the model ID, case-insensitively. This handles:glm-4.5,glm-4.6,glm-4.7mlx-community/GLM-4.5-4bitGLM-4.5-UD-Q8_K_XL-00001-of-00008.ggufTHUDM/glm-4-9b-chatchatglm-6b,chatglm2-6bChanges
New utility (
src/api/providers/utils/glm-model-detection.ts):isGlmModel(modelId)- Detects if a model ID represents a GLM modelgetGlmModelOptions(modelId)- Returns recommended options for GLM modelsLmStudioHandler - Now detects GLM models and applies:
mergeToolResultText: true- Prevents conversation flow disruptionparallel_tool_calls: false- Disabled as GLM may not support itBaseOpenAiCompatibleProvider - Same GLM detection and optimizations
Tests - 41 tests covering various GLM model name formats
Feedback and guidance are welcome!
Important
Introduces GLM model detection utility and updates handlers to apply GLM-specific options, with tests for various model formats.
glm-model-detection.ts):isGlmModel(modelId): Detects GLM models using regex/glm/i.getGlmModelOptions(modelId): Returns options for GLM models, disablingparallel_tool_callsand enablingmergeToolResultText.LmStudioHandlerandBaseOpenAiCompatibleProvidernow usegetGlmModelOptions()to apply GLM-specific options.glm-model-detection.spec.ts):This description was created by
for 0016aa0. You can customize this summary. It will automatically update as commits are pushed.