Merged
Conversation
5600f68 to
18b810a
Compare
Contributor
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed everything up to e181798 in 6 minutes and 0 seconds. Click for details.
- Reviewed
13188lines of code in29files - Skipped
0files when reviewing. - Skipped posting
11draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. frontend/lib/db/migrations/relations.ts:315
- Draft comment:
The relation for 'semanticEventTriggerSpansRelations' only maps 'projectId' even though the schema defines a composite foreign key (projectId and eventDefinitionId). Please update the relation to include both fields. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. frontend/package.json:16
- Draft comment:
The 'schema-pull:lint' script uses inline sed commands (e.g. sed -i.bak) which may not work consistently across different platforms (like Windows). Consider a more cross‑platform solution. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% The diff shows that a newsedcommand was added to an existing script that already usedsed. The comment is technically correct thatsed -i.bakmay have cross-platform issues, but this is not a problem introduced by this PR - it's a pre-existing issue. The comment is about the overall script structure rather than the specific change made (adding the vector-to-jsonb replacement). According to the rules, I should not keep comments about unchanged code or pre-existing issues. The comment would be valid if this PR was introducing thesedusage for the first time, but it's not. The PR does add anothersedcommand, which could be seen as exacerbating an existing cross-platform issue. Perhaps the comment is valid because it's pointing out that the PR is making the problem worse by adding more platform-specific commands. While the PR does add anothersedcommand, the fundamental cross-platform issue already existed. The comment doesn't specifically address the new change but rather critiques the entire approach. If the existingsedcommand was acceptable, then adding one more in the same pattern shouldn't trigger a comment. This is more about the overall script design than the specific change made. The comment is about a pre-existing cross-platform compatibility issue with the script, not specifically about the change made in this PR. Since the script already usedsed -i.bakbefore this change, and the comment doesn't address anything new introduced by the diff, it should be deleted.
3. frontend/lib/db/migrations/0063_flashy_leech.sql:1
- Draft comment:
Typo: The SQL statements are appended with "--> statement-breakpoint". If these are intended as comments, please use proper SQL comment syntax (e.g., '-- statement-breakpoint'). - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% This is an auto-generated migration file. The--> statement-breakpointpattern appears consistently throughout the file, which strongly suggests it's intentional and part of the migration tool's format. The syntax is technically valid SQL (anything after--is a comment). The comment is suggesting this is a typo, but it's almost certainly not - it's the format used by the migration generator. This comment would be asking the author to modify auto-generated code, which doesn't make sense. The comment is not about a logic issue or bug, but rather a misunderstanding of the file format. Could this actually be a typo that was propagated by a misconfigured migration tool? Maybe the tool itself has a bug and this comment is catching it? Even if the migration tool had a bug, this is auto-generated code that shouldn't be manually edited. The proper fix would be to regenerate the migration, not manually edit it. Additionally, the syntax is valid SQL, so it's not causing any functional issues. The comment doesn't provide actionable value. This comment should be deleted. It's flagging auto-generated migration file syntax as a typo when it's actually the intentional format used by the migration tool. The syntax is valid SQL and shouldn't be manually modified.
4. frontend/lib/db/migrations/meta/0062_snapshot.json:1872
- Draft comment:
Typo alert: The foreign key constraint name uses 'labelling' (with double l), while the actual table names use 'labeling' (with a single l). Please confirm if this inconsistency is intentional. - Reason this comment was not posted:
Comment was on unchanged code.
5. frontend/lib/db/migrations/meta/0062_snapshot.json:2700
- Draft comment:
Typo: The foreign key name "semantic_event_trigger_spans_event_definition_id_project_i_fkey" appears to have a missing 'd' in 'project_id'. Please confirm if it should be renamed to "semantic_event_trigger_spans_event_definition_id_project_id_fkey". - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% This is a database migration snapshot file (JSON schema). The comment suggests the foreign key name has a missing 'd' in 'project_id'. However, PostgreSQL has a 63-character limit on identifier names. When a constraint name exceeds this limit, PostgreSQL truncates it. The namesemantic_event_trigger_spans_event_definition_id_project_i_fkeyappears to be truncated at 'project_i' instead of 'project_id'. This is standard PostgreSQL behavior, not a typo. The comment is incorrect - this is not something that needs to be "fixed" or "confirmed" as it's automatically generated by the database system. Could this be an intentional naming convention rather than a truncation? Perhaps the developer deliberately chose this name? However, given the pattern of other foreign key names in the file and the fact this is a generated migration snapshot, it's almost certainly an automatic truncation. While it's theoretically possible this was intentional, the context strongly suggests this is a generated migration file where PostgreSQL automatically truncated the constraint name due to length limits. This is standard database behavior and not something the PR author needs to address or confirm. The comment is asking them to "confirm" something that is automatically handled by the database. This comment should be deleted. The foreign key name truncation is standard PostgreSQL behavior due to the 63-character identifier limit, not a typo. This is a generated migration snapshot file, and the naming is automatic. The comment incorrectly suggests this needs to be fixed or confirmed.
6. frontend/lib/db/migrations/meta/0063_snapshot.json:1658
- Draft comment:
Typographical consistency issue: The foreign key is named 'labelling_queue_items_queue_id_fkey' (with 'labelling' using double 'l') while the table name in 'tableFrom' is 'labeling_queue_items' (with a single 'l'). Consider standardizing the spelling across the schema. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 15% vs. threshold = 50% This is a new file being added (the entire file is new based on the diff showing it starts with "new file"). The comment is about an internal inconsistency in the schema - a foreign key name uses British spelling while the table uses American spelling. However, this appears to be an auto-generated migration snapshot file. These files are typically generated by migration tools and shouldn't be manually edited. The fix would need to happen in the actual migration SQL files or schema definitions, not in this snapshot. Additionally, while it's a valid observation about inconsistency, it's not clear this would cause any functional issues - it's purely a naming convention concern. The comment doesn't provide actionable guidance on what to change. This file appears to be auto-generated by a migration tool, so comments about its contents may not be actionable since developers typically don't manually edit these snapshot files. The inconsistency might have originated from an earlier migration that used British spelling, and fixing it would require a new migration rather than editing this snapshot. While the file is likely auto-generated, the comment could still be valuable if it prompts the developer to create a migration to fix the naming inconsistency. However, the comment doesn't clearly indicate this is what should be done, and naming inconsistencies in foreign key constraints (while not ideal) don't typically cause functional problems - they're purely cosmetic issues. This comment should be deleted. It's pointing out a naming inconsistency in an auto-generated database snapshot file, which cannot and should not be manually edited. If the inconsistency needs to be fixed, it would require a new migration, but the comment doesn't make this clear and doesn't provide actionable guidance. Additionally, this is a minor cosmetic issue that doesn't affect functionality.
7. frontend/lib/db/migrations/meta/0064_snapshot.json:1658
- Draft comment:
Typographical error: The foreign key name is 'labelling_queue_items_queue_id_fkey' while elsewhere the table is named 'labeling_queue_items' (with one 'l'). Consider using consistent spelling (e.g., 'labeling_queue_items_queue_id_fkey'). - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% This is a migration snapshot file, which is typically auto-generated by database migration tools like Drizzle. The comment identifies a real inconsistency - the foreign key name uses "labelling" (British spelling) while the table uses "labeling" (American spelling). However, this file is a snapshot/metadata file that is likely generated automatically from the actual database schema. The real fix would need to happen in the source schema definition or migration files, not in this snapshot. Additionally, while the inconsistency exists, it's purely cosmetic - the foreign key will still function correctly. The comment doesn't provide actionable guidance on where to fix this, and modifying auto-generated files is generally not recommended. This could be a legitimate issue if the team cares about naming consistency, and it might indicate an inconsistency in the actual database schema that should be fixed at the source. The comment does identify a real inconsistency that exists in the file. While the inconsistency is real, this is an auto-generated snapshot file. Comments on auto-generated files are generally not useful because: 1) The file shouldn't be manually edited, 2) The fix needs to happen elsewhere (in the source schema), and 3) The tool flagged this as a new file addition, so this isn't a change the author made - it's just the current state of the database. The comment also doesn't clearly indicate that this needs to be fixed in a migration or schema file, making it not actionable. This comment should be deleted. It's commenting on an auto-generated database snapshot file where manual edits are not appropriate. The inconsistency, while real, is cosmetic and doesn't affect functionality. If this needs to be fixed, it should be done through a proper database migration, not by editing this snapshot file.
8. frontend/lib/db/migrations/relations.ts:3
- Draft comment:
There's a minor typographical issue in the import list on this line: 'traces,tracesAgentChats'. A space after the comma (i.e. 'traces, tracesAgentChats') would improve readability and consistency. - Reason this comment was not posted:
Comment was on unchanged code.
9. frontend/lib/db/migrations/schema.ts:403
- Draft comment:
Typographical inconsistency: The foreign key name is "labelling_queue_items_queue_id_fkey" using 'labelling' while the table is defined as "labeling_queue_items". Consider using consistent spelling (either 'labeling' or 'labelling') throughout. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% This comment identifies a legitimate inconsistency in naming conventions. The table uses American spelling 'labeling' but the foreign key constraint uses British spelling 'labelling'. However, I need to check if this is actually a change made in this diff. Looking at the diff, thelabelingQueueItemstable appears in the new version but I need to see if the foreign key name was changed. In the diff, I can see the table moved positions but the foreign key namelabelling_queue_items_queue_id_fkeyappears to be unchanged from the original. This appears to be pre-existing code that was just moved around in the file reorganization. The comment is about code that wasn't actually changed in this diff - it's just been moved to a different location in the file. While the inconsistency is real and the comment is technically correct, the rule states I should only keep comments about changes made by the diff. This appears to be existing code that was simply moved to a different location in the file. The foreign key name itself wasn't modified in this diff. Looking more carefully at the diff, the labelingQueueItems table and its foreign key definition appear in both the old and new versions. The diff shows this is a reorganization/reordering of table definitions, not a change to the foreign key name itself. Since the inconsistency was pre-existing and not introduced by this diff, the comment should be deleted per the rules. This comment should be deleted because it's about pre-existing code that wasn't changed in this diff. The spelling inconsistency between 'labeling' and 'labelling' existed before this change, and the diff only moved the table definition to a different location in the file without modifying the foreign key name.
10. frontend/lib/db/migrations/schema.ts:680
- Draft comment:
Typo: The unique constraint name 'label_classes_name_project_id_unique' seems incorrectly named since the table is 'tag_classes'. Did you mean to use 'tag_classes_name_project_id_unique' instead? - Reason this comment was not posted:
Marked as duplicate.
11. frontend/lib/db/migrations/schema.ts:765
- Draft comment:
The unique index name "traces_project_id_id_unique" looks like it may contain a typo with a duplicated 'id'. Please verify if it should be named "traces_project_id_unique" or if this duplication is intentional. - Reason this comment was not posted:
Comment looked like it was already resolved.
Workflow ID: wflow_SwL9n3qOaVQNdA3I
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
Contributor
There was a problem hiding this comment.
Important
Looks good to me! 👍
Reviewed 30868b6 in 1 minute and 52 seconds. Click for details.
- Reviewed
55lines of code in2files - Skipped
0files when reviewing. - Skipped posting
2draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. frontend/app/api/projects/[projectId]/evaluation-scores/[evaluationResultId]/route.ts:45
- Draft comment:
Good update: projectId is now extracted and passed to updateEvaluationScore. Ensure the projectId is validated elsewhere as a proper UUID if needed. - Reason this comment was not posted:
Confidence changes required:33%<= threshold50%None
2. frontend/lib/actions/evaluation-score/index.ts:34
- Draft comment:
The SQL query aliases the column as 'score' (using 'value score'), but the result is cast as an object with property 'value'. Consider casting to { score: number } to match the alias. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
Workflow ID: wflow_7BZIHSQxEUM6FwJo
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
Contributor
|
Generated with ❤️ by ellipsis.dev |
| name, | ||
| score, | ||
| }; | ||
| } |
There was a problem hiding this comment.
Bug: Silent failure when updating non-existent evaluation score
The updateEvaluationScore function lost error handling when migrating from PostgreSQL to ClickHouse. The old PostgreSQL version checked if the update affected a row and threw "Evaluation score not found" if not. The new ClickHouse version uses ALTER TABLE UPDATE which silently affects zero rows if the score doesn't exist, then returns a fabricated success object. This means users see "Score saved successfully" in the UI even when nothing was actually saved, creating misleading feedback during human evaluation scoring.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Moves evaluation scores from PostgreSQL to ClickHouse and updates backend, APIs, frontend, and migrations to match the new schema and data flow.
update_eval_datapoint: stop writing scores to PostgreSQL, keep updatingexecutor_output, and write scores to ClickHouse (evaluation_scores, datapoint outputs).set_evaluation_results: bulk insert/updateevaluation_resultsonly (no scores); utils/columns simplified (removedscores).save_evaluation_scores: parallel inserts to ClickHouse for scores/datapoints; DB write only for results.getEvaluationTimeProgression(projectId, groupId, aggregate, ids); URL-encodegroupId.projectId; read/update scores via ClickHouse (getEvaluationScore,updateEvaluationScore).human-evaluation-scoreremoves global SWR mutate; invitations table filters null emails; cluster columns minor accessor cleanup.agent_*,clusters,event_*,slack_*,dataset_export_jobs,dataset_parquets,project_settings,workspace_deployments.dataset_datapoints,evaluation_scores,spans,summary_trigger_spans,tags,traces_summaries,user_*usage,workspace_usage.tracesPK to(id, project_id)and indexes; adjust defaults/nullability (project_api_keys,playgrounds,event_clusters).schema.ts,relations.ts, snapshots; update script to replacevector()withjsonb().Written by Cursor Bugbot for commit 9f4e24f. This will update automatically on new commits. Configure here.
Important
Overhauled database schema and updated backend/frontend to move evaluation scores from PostgreSQL to ClickHouse, affecting multiple components.
agent_*,clusters,event_*,slack_*,dataset_export_jobs,dataset_parquets,shared_*,project_settings.dataset_datapoints,evaluation_scores,spans,summary_trigger_spans,tags,traces_summaries,user_*,workspace_usage.tracesprimary key and indexes.project_api_keys,playgrounds,event_clusters.schema.ts,relations.ts, and migration snapshots.update_evaluation_datapoint_and_get_trace_id.getEvaluationTimeProgressionAPI, removed time range.projectIdand read from ClickHouse.human-evaluation-score.tsxandinvitations-table.tsx.human-evaluation-score.tsx.This description was created by
for 30868b6. You can customize this summary. It will automatically update as commits are pushed.