Summary

• Make the hardcoded token repeat limit (30) configurable via new token_repeat_limit option (default: 0 = disabled)
• Fix incorrect done: false response when repeat limit is hit — now properly sends done: true with done_reason: "repeat"
• Skip empty/whitespace-only tokens in repeat tracking to prevent false positives
• Map "repeat" to "stop" in OpenAI compatibility layer for spec compliance

Context

The hardcoded repeat limit of 30 tokens in llm/server.go caused false positives on valid model output (e.g., OCR dots . . . . . . . as reported in #14117). When triggered, the function returned ctx.Err() without sending a final done: true response, leaving clients in an ambiguous state.

Changes

Usage

# Via API option
curl http://localhost:11434/api/generate -d '{
  "model": "glm-ocr",
  "prompt": "OCR this image",
  "options": {"token_repeat_limit": 100}
}'

# Via Modelfile
PARAMETER token_repeat_limit 100

Test plan

• Verify token_repeat_limit: 0 (default) disables repeat detection — no false positives on repeated dots/patterns
• Verify token_repeat_limit: 100 stops generation after 100 consecutive identical tokens
• Verify response includes done: true and done_reason: "repeat" when limit is hit
• Verify OpenAI /v1/chat/completions endpoint returns finish_reason: "stop" (not "repeat")
• Verify existing tests pass

Fixes #14117