Expand dataset builder in the UI #5156
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull request overview
This PR restructures the dataset builder in the UI to use the new /v1/datasets/{dataset_name}/from_inferences API endpoint. The changes consolidate filtering parameters, deprecate old API types, and expand the dataset builder UI to include inference filtering capabilities.
Key Changes:
- Renamed
filtertofiltersacross the codebase for API consistency - Restructured the
CreateDatapointsFromInferenceRequestAPI to flatten theListInferencesRequestparameters into theInferenceQueryvariant - Consolidated
CreateDatapointsFromInferenceOutputSourceinto the existingInferenceOutputSourceenum
Reviewed changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| ui/e2e_tests/observability.inferences.filter.spec.ts | Updated test to use getByLabel instead of getByPlaceholder for finding the Episode ID input |
| ui/app/utils/tensorzero/tensorzero.ts | Added createDatapointsFromInferences client method and inlined EvaluationRunStatsResponse type definition |
| ui/app/utils/clickhouse/inference.server.ts | Renamed parameter from filter to filters for consistency |
| ui/app/routes/observability/inferences/route.tsx | Renamed URL parameter and loader data from filter to filters |
| ui/app/routes/observability/inferences/InferencesTable.tsx | Updated component to use filters prop and added htmlFor attributes for accessibility |
| ui/app/routes/datasets/builder/types.ts | Replaced metric-based filtering with inference filtering parameters (variant_name, episode_id, search_query, filters) |
| ui/app/routes/datasets/builder/route.tsx | Restructured to use the new createDatapointsFromInferences API with inference_query type |
| ui/app/routes/datasets/builder/OutputSourceSelector.tsx | Removed demonstration count fetching and UI complexity |
| ui/app/routes/datasets/builder/DatasetCountDisplay.tsx | Deleted file - count display is no longer needed with the new API |
| ui/app/routes/datasets/builder/DatasetBuilderForm.tsx | Expanded UI to include variant, episode ID, search query, and advanced filter builder inputs |
| ui/app/routes/api/datasets/count_inserts.route.ts | Deleted file - row counting endpoint is no longer needed |
| ui/app/routes.ts | Removed the count_inserts route |
| tensorzero-core/tests/e2e/endpoints/datasets/create_from_inferences.rs | Updated tests to use the new flattened ListInferencesRequest structure |
| tensorzero-core/tests/e2e/endpoints/datasets/clone_datapoints.rs | Updated imports and removed usage of deprecated CreateDatapointsFromInferenceOutputSource |
| tensorzero-core/src/stored_inference.rs | Changed into_datapoint_insert to use InferenceOutputSource instead of CreateDatapointsFromInferenceOutputSource |
| tensorzero-core/src/endpoints/stored_inferences/v1/types.rs | Added filters field and deprecated filter field with backward compatibility handling |
| tensorzero-core/src/endpoints/datasets/v1/types.rs | Removed CreateDatapointsFromInferenceOutputSource, restructured request to flatten ListInferencesRequest |
| tensorzero-core/src/endpoints/datasets/v1/create_from_inferences.rs | Refactored to use flattened ListInferencesRequest and InferenceOutputSource |
| tensorzero-core/src/endpoints/datasets/legacy.rs | Renamed handler and added deprecation warning for the legacy endpoint |
| tensorzero-core/src/db/inferences.rs | Added None variant to InferenceOutputSource enum |
| tensorzero-core/src/db/clickhouse/inference_queries.rs | Updated query generation to handle InferenceOutputSource::None |
| internal/tensorzero-node/lib/bindings/index.ts | Removed export of deprecated CreateDatapointsFromInferenceOutputSource |
| internal/tensorzero-node/lib/bindings/ListInferencesRequest.ts | Renamed filter field to filters |
| internal/tensorzero-node/lib/bindings/InferenceOutputSource.ts | Added "none" as a possible value |
| internal/tensorzero-node/lib/bindings/CreateDatapointsFromInferenceRequestParams.ts | Flattened ListInferencesRequest fields into the InferenceQuery variant |
| internal/tensorzero-node/lib/bindings/CreateDatapointsFromInferenceRequest.ts | Restructured to flatten parameters and move output_source into variant-specific fields |
| internal/tensorzero-node/lib/bindings/CreateDatapointsFromInferenceOutputSource.ts | Deleted file - type consolidated into InferenceOutputSource |
| gateway/src/routes/internal.rs | Updated to use renamed deprecated_create_datapoints_from_inferences_handler |
| clients/rust/tests/test_stored_inferences.rs | Renamed filter to filters in test |
| clients/rust/tests/test_datasets.rs | Updated to use flattened ListInferencesRequest structure |
| clients/rust/src/lib.rs | Removed CreateDatapointsFromInferenceOutputSource export and output_source parameter from method signature |
| clients/python/tensorzero/generated_types.py | Updated Python type definitions to match new API structure |
| clients/python/src/lib.rs | Removed output_source parameter from Python client method signatures |
…gb/dataset-builder
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Member
Author
|
@codex review |
Member
Author
|
/merge-queue |
Contributor
|
🚀 Merge queue workflow triggered! View the run: https://github.com/tensorzero/tensorzero/actions/runs/20273251484 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
virajmehta
reviewed
Dec 17, 2025
virajmehta
requested changes
Dec 17, 2025
Member
virajmehta
left a comment
There was a problem hiding this comment.
generally l think this LGTM; the copilot nit is something we should fix though.
…gb/dataset-builder
virajmehta
approved these changes
Dec 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
Enhances dataset builder UI to create datasets from inferences with advanced filtering and adds backend support for these features.
DatasetBuilderFormto create datasets from inferences with filters like function, variant, episode ID, and search query.VariantSelectorfor selecting function variants.OutputSourceSelectorto choose output source for dataset creation.DatasetBuilderFormusinguseInferenceCountFetcher.countInferencesAPI endpoint to count inferences matching query parameters.createDatapointsFromInferencesto support new filtering options.TensorZeroClientto interact with new API endpoints.datasets.builder.spec.tsfor dataset creation with various filters.observability.inferences.filter.spec.tsfor inference filtering UI.This description was created by
for 7e7a519. You can customize this summary. It will automatically update as commits are pushed.