Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: metrics schema version #12732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Jun 17, 2025
Merged

feat: metrics schema version #12732

merged 13 commits into from
Jun 17, 2025

Conversation

vittoriopolverino
Copy link
Member

@vittoriopolverino vittoriopolverino commented Jun 9, 2025 •
edited
Loading

Motivation

This PR introduces explicit schema versioning for metrics definitions. The goal is to make telemetry changes safer and more maintainable:

  • enabling easier debugging and auditability of metric changes over time
  • makes it easier to analyze data across schema versions by maintaining backward compatibility

For now:.

  • No persistent file or central registry to verify schema consistency
  • No strict validation

However, having the schema_version available in Tinybird helps with:

  • Managing backward compatibility.
  • Reconciling metrics whose schema may have evolved over time.
    • For example, suppose an initial version of a counter uses label_1_value to store the status of a request, with possible values like "success" and "error". Later, the counter is updated: a new label label_2 is introduced to hold the status instead:
multiIf(
      schema_version = 1, label_1_value,
      schema_version = 2, label_6_value,
      schema_version = 3, label_*_value,
      ...
  ) AS status

A more structured approach would require:

  • A schema registry.
  • Explicit contracts for each metric.
  • Validation at build time, CI, or runtime to ensure consistency between contracts and the metric definitions in code.

Changes

  • Introduced a naive approach for handling schema_version in metrics.

@vittoriopolverino vittoriopolverino self-assigned this Jun 9, 2025
@vittoriopolverino vittoriopolverino added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Jun 9, 2025
@vittoriopolverino vittoriopolverino added this to the 4.6 milestone Jun 9, 2025
@vittoriopolverino vittoriopolverino mentioned this pull request Jun 9, 2025
Merged
@vittoriopolverino vittoriopolverino force-pushed the DAT-156-metrics-schema-version branch from 5025e2a to 8fdc9e8 Compare June 12, 2025 13:35
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results - Preflight, Unit

21 630 tests  +9   19 973 ✅ +9   6m 13s ⏱️ -22s
     1 suites ±0    1 657 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit ea2ffc8. ± Comparison against base commit 21c4d5d.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 8s ⏱️ -7s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit ea2ffc8. ± Comparison against base commit 21c4d5d.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results - Alternative Providers

987 tests   589 ✅  29m 44s ⏱️
  4 suites  398 💤
  4 files      0 ❌

Results for commit ea2ffc8.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 21m 16s ⏱️
5 231 tests 4 299 ✅ 929 💤 3 ❌
5 237 runs  4 299 ✅ 935 💤 3 ❌

For more details on these failures, see this check.

Results for commit ea2ffc8.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files      2 suites   1h 42m 38s ⏱️
4 874 tests 4 094 ✅ 777 💤 3 ❌
4 876 runs  4 094 ✅ 779 💤 3 ❌

For more details on these failures, see this check.

Results for commit ea2ffc8.

♻️ This comment has been updated with latest results.

@vittoriopolverino vittoriopolverino marked this pull request as ready for review June 12, 2025 15:22
@vittoriopolverino vittoriopolverino requested a review from thrau as a code owner June 12, 2025 15:22
@vittoriopolverino vittoriopolverino marked this pull request as draft June 12, 2025 15:23
@vittoriopolverino vittoriopolverino marked this pull request as ready for review June 12, 2025 18:56
alexrashed
alexrashed approved these changes Jun 16, 2025
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, and in my opinion you should just define this on the base Metric class from the beginning. However, I'll totally leave this up to you! Thanks for continuously improving the metrics framework! 💯

@vittoriopolverino vittoriopolverino marked this pull request as draft June 16, 2025 18:54
@vittoriopolverino vittoriopolverino marked this pull request as ready for review June 16, 2025 19:26
@vittoriopolverino vittoriopolverino added the review: merge when ready Signals to the reviewer that a PR can be merged if accepted label Jun 17, 2025
@alexrashed alexrashed force-pushed the DAT-156-metrics-schema-version branch from c475ccf to ea2ffc8 Compare June 17, 2025 09:39
@alexrashed
Copy link
Member

FYI: The three failing tests are unrelated / are also failing on master right now and will be resolved with #12767.

@alexrashed alexrashed merged commit 1d3ef0c into master Jun 17, 2025
37 of 41 checks passed
@alexrashed alexrashed deleted the DAT-156-metrics-schema-version branch June 17, 2025 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexrashed alexrashed alexrashed approved these changes

@thrau thrau Awaiting requested review from thrau thrau is a code owner

Assignees

@vittoriopolverino vittoriopolverino

Labels
review: merge when ready Signals to the reviewer that a PR can be merged if accepted semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Milestone
4.6
Development

Successfully merging this pull request may close these issues.
2 participants
@vittoriopolverino @alexrashed