fix(context_compaction): charge inline base64 tool-result images at flat image cost - #29818
fix(context_compaction): charge inline base64 tool-result images at flat image cost#29818Timmy-South wants to merge 1 commit into
Conversation
|
Notice regarding contribution eligibility This pull request is missing the required checked confirmation:
This pull request has been closed. If this statement applies to your contribution, check the box ( |
|
Notice regarding Contributor License Agreement The description of this pull request does not contain the necessary Contributor License Agreement (CLA) text. The inclusion of the exact CLA confirmation text is a prerequisite for the review and acceptance of any contribution. Without this text, the contribution cannot be evaluated. This pull request has been closed. To proceed, please open a new pull request utilizing the provided PR template, which includes the requisite legal text. |
Summary
_estimate_messages_tokenscharged a message'sfileslist through_estimate_tokens, which JSON-serializes the value and countslen // 4. Tool-result images are persisted inline as base64 data URIs, so this counted the entire base64 payload as text tokens.For a single ~3 MB image (≈ 4,197,680 base64 chars) this produced a ~1,049,420-token estimate for one message — larger than the backend's whole KV pool — and fired spurious auto-compaction even though engine-side prompt tokens stayed normal.
This PR:
filesentries that carry adata:image URI with the flat image cost already used for content images (image/image_url→ 1000 tokens), andWhy
Reported in #29761. #29765 fixes the model-facing leak (
middleware.pyextract_base64_images) but leaves this estimator charging the same base64 viafiles, so the spurious compaction and inflated context meter persisted.How I verified
Standalone reproduction using the reported scenario (one
filesentry with a ~4,197,680-char base64 data URI):Notes
backend/open_webui/utils/context_compaction.pyis touched.files.py,middleware.py).