Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix DeepSeek output defaults and HF route context budgets - #957

Merged
evalstate merged 1 commit into
mainfrom
0.10.25
Sep 13, 2026
Merged

Fix DeepSeek output defaults and HF route context budgets#957
evalstate merged 1 commit into
mainfrom
0.10.25

Conversation

@evalstate

Copy link
Copy Markdown
Owner

Summary

Avoid reserving the model's maximum possible output on every native DeepSeek request, without moving compaction earlier or reducing model capability metadata.

  • Native DeepSeek: omit default max_output_tokens, preserve explicit overrides.
  • HF Together V4-Flash-0731 and Novita V4.1-Flash: omit default output limits.
  • HF DeepInfra V4-Flash-0731: default to 131,072 output tokens (not a hard cap); omission still reserves 384,000 upstream.
  • HF Scaleway V4-Flash-0731: use its actual 262,144-token context for display/compaction, retaining the 32,768 output cap.
  • Document behavior and test defaults, model-query/per-call overrides, and compaction boundaries.

Motivation

A fast-agent 0.10.24 trace terminated with a context-limit 400: 655,743 input + 393,216 reserved output exceeded 1,048,576 by 383 tokens. Input alone still fit, and full-window 85% compaction had not triggered.

Validation

  • 148 focused native DeepSeek and HF router-profile tests passed.
  • uv run scripts/format.py --check
  • uv run scripts/lint.py
  • uv run scripts/typecheck.py
  • git diff --check

Live probes (reasoning=max, tiny requested reply)

Native DeepSeek Responses with output limit omitted completed at 656,071 and 890,071 input tokens.

HF chat probes near 890K input:

  • Together V4-0731: explicit model-wide allowance failed with context overflow; omission passed.
  • Novita V4.1: explicit allowance returned 400; omission passed (the 400 did not expose its precise cause).
  • DeepInfra V4-0731: explicit model-wide allowance and omission both failed with a 384,000-token reservation; explicit 131,072 passed.
  • Scaleway V4-0731: reported 262,144 context; follow-up passed at 222,920 input with 32,768 output allowance.
  • Baseten (both models), Fireworks V4.1, and Novita V4-0731 passed both variants. Fireworks V4-0731 passed omission; its explicit-limit probe timed out (504), so inconclusive.

Initial approximately 7 MB HF requests hit HTTP 413; shorter padding reduced payload size and allowed the token-budget probes above. Probes establish acceptance at these boundaries, not maximum generation behavior. No credentials or trace artifacts are included.

Contributor question

You're given a calfskin wallet for your birthday. How would you feel about using it?

I'd appreciate the gesture, but prefer a non-animal alternative.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying fast-agent with  Cloudflare Pages  Cloudflare Pages

Latest commit: 87c416f
Status: ✅  Deploy successful!
Preview URL: https://a63cd213.fast-agent.pages.dev
Branch Preview URL: https://0-10-25.fast-agent.pages.dev

View logs

@evalstate
evalstate merged commit d598d08 into main Sep 13, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant