Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(groq): add prompt caching token usage details #33708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

feat(groq): add prompt caching token usage details #33708

wants to merge 5 commits into from
+669 −18

Conversation

@MshariAlaeena
Copy link

@MshariAlaeena MshariAlaeena commented Oct 28, 2025

Description:
Adds support for prompt caching usage metadata in ChatGroq. The integration now captures cached token information from the Groq API response and includes it in the input_token_details field of the usage_metadata.

Changes:

  • Created new _create_usage_metadata() helper function to centralize usage metadata creation logic
  • Extracts cached_tokens from prompt_tokens_details in API responses and maps to input_token_details.cache_read
  • Integrated the helper function in both streaming (_convert_chunk_to_message_chunk) and non-streaming (_create_chat_result) code paths
  • Added comprehensive unit tests to verify caching metadata handling and backward compatibility

This enables users to monitor prompt caching effectiveness when using Groq models with prompt caching enabled.

Issue: N/A

Dependencies: None

@github-actions github-actions bot added integration Related to a provider partner package integration feature labels Oct 28, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented Oct 28, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #33708 will not alter performance

Comparing MshariAlaeena:malaeena/add-prompt-caching-usage-fields (469e62d) with master (dfb05a7)1

Summary

✅ 1 untouched
⏩ 33 skipped2

Footnotes

  1. No successful run was found on master (c6547f5) during the generation of this report, so dfb05a7 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

  2. 33 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

ccurme
ccurme approved these changes Oct 28, 2025
View reviewed changes
Copy link
Collaborator

@ccurme ccurme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ccurme ccurme ccurme approved these changes

@mdrxy mdrxy Awaiting requested review from mdrxy mdrxy is a code owner

Assignees

No one assigned

Labels

feature integration Related to a provider partner package integration

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MshariAlaeena @ccurme