Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@shuyangli
Copy link
Member

@shuyangli shuyangli commented Dec 17, 2025

  • Added a new internal API endpoint /internal/functions/{function_name}/throughput-by-variant that returns inference counts grouped by variant and time period.
  • Also add a serde helper to format datetime in the RFC 3339 format with millis: 2025-12-01T13:10:11.000Z

Important

Add /internal/functions/{function_name}/throughput-by-variant endpoint to fetch function throughput data grouped by variant and time period, with tests and datetime serialization support.

  • Behavior:
    • Adds /internal/functions/{function_name}/throughput-by-variant endpoint in internal.rs to return inference counts grouped by variant and time period.
    • Supports time windows: minute, hour, day, week, month, cumulative.
  • Models:
    • Adds GetFunctionThroughputByVariantParams and VariantThroughput to inference_stats.rs.
    • Adds GetFunctionThroughputByVariantResponse and VariantThroughput types in GetFunctionThroughputByVariantResponse.ts and VariantThroughput.ts.
  • Functions:
    • Implements get_function_throughput_by_variant in inference_stats.rs.
    • Adds serialize_utc_datetime_rfc_3339_with_millis to serde_util.rs for datetime serialization.
  • Tests:
    • Adds tests for the new endpoint in inference_stats_queries.rs and inference_stats.rs.
    • Adds E2E tests in inference_stats.rs for various time windows and edge cases.
  • Misc:
    • Updates Cargo.toml and Cargo.lock to include regex dependency.

This description was created by Ellipsis for 341c0a8. You can customize this summary. It will automatically update as commits are pushed.

@shuyangli shuyangli mentioned this pull request Dec 17, 2025
52 tasks
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from aaaebb2 to 05172b4 Compare December 17, 2025 16:03
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch 5 times, most recently from a6f8e3f to ff46d32 Compare December 17, 2025 20:05
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from ff46d32 to 5496675 Compare December 17, 2025 22:41
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 5496675 to 5568427 Compare December 18, 2025 15:25
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 5568427 to 4a3c442 Compare December 18, 2025 16:37
@shuyangli shuyangli marked this pull request as ready for review December 18, 2025 16:40
Copy link
Member Author

/merge-queue

@github-actions
Copy link
Contributor

🚀 Merge queue workflow triggered!

View the run: https://github.com/tensorzero/tensorzero/actions/runs/20344301638

@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 4a3c442 to 2f91657 Compare December 18, 2025 16:57
@shuyangli shuyangli enabled auto-merge December 18, 2025 16:59
@shuyangli
Copy link
Member Author

/merge-queue

@github-actions
Copy link
Contributor

🚀 Merge queue workflow triggered!

View the run: https://github.com/tensorzero/tensorzero/actions/runs/20345730523

@virajmehta virajmehta assigned shuyangli and unassigned virajmehta Dec 18, 2025
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 2f91657 to c604a6d Compare December 18, 2025 18:38
@shuyangli shuyangli assigned shuyangli and virajmehta and unassigned shuyangli Dec 18, 2025
virajmehta
virajmehta previously approved these changes Dec 18, 2025
@shuyangli shuyangli added this pull request to the merge queue Dec 18, 2025
@shuyangli shuyangli removed this pull request from the merge queue due to a manual request Dec 18, 2025
@shuyangli shuyangli added this pull request to the merge queue Dec 18, 2025
github-merge-queue bot pushed a commit that referenced this pull request Dec 18, 2025
* Migrate getFunctionThroughputByVariant

* Use idiomatic Rust Duration and ClickHouse UUIDv7ToDateTime for time window filtering

Refactors the time-based filtering in ClickHouse inference stats queries:

- Changed time_window_to_ms to time_window_to_duration returning std::time::Duration
- Use ClickHouse's UUIDv7ToDateTime function for timestamp comparison instead of
  manual bit manipulation of UUIDv7 format
- Pass time delta in seconds to ClickHouse, use INTERVAL SECOND for arithmetic
- Preserves original semantics of filtering relative to max timestamp in the data

Fix inference stats e2e test assertions

- Check RFC 3339 format with milliseconds on raw HTTP response body instead of
  re-serializing DateTime (chrono's default serializer omits .000 for zero ms)
- Relax variant count assertion from 2+ to 1+ for fixture data flexibility
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Dec 18, 2025
Copy link
Member Author

/merge-queue

@github-actions
Copy link
Contributor

🚀 Merge queue workflow triggered!

View the run: https://github.com/tensorzero/tensorzero/actions/runs/20351499551

@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 0594a2a to 341c0a8 Compare December 18, 2025 21:32
@shuyangli
Copy link
Member Author

/merge-queue

@github-actions
Copy link
Contributor

🚀 Merge queue workflow triggered!

View the run: https://github.com/tensorzero/tensorzero/actions/runs/20351984757

@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 341c0a8 to c329576 Compare December 18, 2025 21:55
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from c329576 to 32e9aee Compare December 18, 2025 22:39
…window filtering

Refactors the time-based filtering in ClickHouse inference stats queries:

- Changed time_window_to_ms to time_window_to_duration returning std::time::Duration
- Use ClickHouse's UUIDv7ToDateTime function for timestamp comparison instead of
  manual bit manipulation of UUIDv7 format
- Pass time delta in seconds to ClickHouse, use INTERVAL SECOND for arithmetic
- Preserves original semantics of filtering relative to max timestamp in the data

Fix inference stats e2e test assertions

- Check RFC 3339 format with milliseconds on raw HTTP response body instead of
  re-serializing DateTime (chrono's default serializer omits .000 for zero ms)
- Relax variant count assertion from 2+ to 1+ for fixture data flexibility
@shuyangli shuyangli force-pushed the sl/migrate-get-function-throughput-by-variant branch from 32e9aee to e538951 Compare December 18, 2025 22:41
@shuyangli shuyangli added this pull request to the merge queue Dec 18, 2025
Merged via the queue into main with commit 354508c Dec 19, 2025
54 checks passed
@shuyangli shuyangli deleted the sl/migrate-get-function-throughput-by-variant branch December 19, 2025 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants