feat:surrealDB queries 39-23 #66

lveillard · 2024-05-23T21:20:51Z

No description provided.

sweep-ai-deprecated · 2024-05-23T21:31:08Z

Sweep: PR Review

This pull request enhances SurrealDB query handling and introduces benchmark testing.

The main change involved modifying how $filter mappings are processed. The $filterProcessed property was removed from various types and functions, and the mapFilterKeys function was relocated to be used exclusively within TQL (Typed Query Language) queries. This ensures that $filter mappings are now only applied in TQL contexts, simplifying the BQL (Basic Query Language) processing logic.

Additionally, a new benchmark test was added. The bench.sh script was created to automate the setup and teardown of a SurrealDB Docker container for running these benchmarks. This script initializes the database, imports schema and data, and executes the benchmarks using npx vitest bench.

Minor changes included updating test descriptions and filter conditions to improve clarity and accuracy. For example, test cases were renamed to better reflect their purpose, such as changing "ef6[entity,filter] - $filter by unique field" to "ef6[entity,filter,id] - $filter by id in filter".

The changelog was updated to reflect these changes, documenting the new version 0.10.13 and the addition of benchmark tests and SurrealDB query enhancements.

Sweep Found These Issues

src/adapters/surrealDB/types/base.ts

The removal of the $filterProcessed property may cause functional issues in parts of the codebase that rely on this property to determine the processing state of filters.

blitz-orm/src/adapters/surrealDB/types/base.ts

Lines 16 to 19 in 1f80eb4

    
           	$var: string; 
        
           	$fieldType: string; 
        
           	$filter?: Record<string, string>; 
        
           };

View Diff

src/stateMachine/query/bql/enrich.ts

Directly assigning field.$filter without transformation in createRoleField may lead to incorrect filter application if the schema requires key mapping.

blitz-orm/src/stateMachine/query/bql/enrich.ts

Lines 305 to 308 in 1f80eb4

    
           $intermediary: relation, 
        
           $justId, 
        
           $id: field.$id, 
        
           $filter: field.$filter,

View Diff

src/stateMachine/query/tql/build.ts

The mapFilterKeys function does not handle cases where thingSchema is null or undefined, which could lead to runtime errors.

blitz-orm/src/stateMachine/query/tql/build.ts

Lines 295 to 315 in 1f80eb4

    
           const mapFilterKeys = (filter: Filter, thingSchema: EnrichedBormEntity | EnrichedBormRelation) => { 
        
           	const mapper: Record<string, string> = {}; 
        
           	thingSchema.dataFields?.forEach((df) => { 
        
           		if (df.path !== df.dbPath) { 
        
           			mapper[df.path] = df.dbPath; 
        
           		} 
        
           	}); 
        
           	if (Object.keys(mapper).length === 0) { 
        
           		return filter; 
        
           	} 
        
           	const { $not, ...f } = filter; 
        
           	const newFilter: Filter = mapPositiveFilterKeys(f, mapper); 
        
           	if ($not) { 
        
           		newFilter.$not = mapPositiveFilterKeys($not as PositiveFilter, mapper); 
        
           	} 
        
           	return newFilter;

View Diff

The buildFilter function now relies on mapFilterKeys, but there is no error handling if mapFilterKeys returns an invalid or unexpected filter structure.

blitz-orm/src/stateMachine/query/tql/build.ts

Lines 327 to 336 in 1f80eb4

    
           const buildFilter = (props: { 
        
           	$filter: Filter; 
        
           	$var: string; 
        
           	$thing: string; 
        
           	schema: EnrichedBormSchema; 
        
           	depth: number; 
        
           }) => { 
        
           	const { $filter: $nonMapedFilter, $var, $thing, schema, depth } = props; 
        
           	const $filter = mapFilterKeys($nonMapedFilter, getThing(schema, $thing));

View Diff

Sweep has identified a redundant function: The new function mapFilterKeys is redundant as its functionality is already covered by processFilter and mapPositiveFilterKeys.

blitz-orm/src/stateMachine/query/tql/build.ts

Lines 295 to 315 in 1f80eb4

    
           const mapFilterKeys = (filter: Filter, thingSchema: EnrichedBormEntity | EnrichedBormRelation) => { 
        
           	const mapper: Record<string, string> = {}; 
        
           	thingSchema.dataFields?.forEach((df) => { 
        
           		if (df.path !== df.dbPath) { 
        
           			mapper[df.path] = df.dbPath; 
        
           		} 
        
           	}); 
        
           	if (Object.keys(mapper).length === 0) { 
        
           		return filter; 
        
           	} 
        
           	const { $not, ...f } = filter; 
        
           	const newFilter: Filter = mapPositiveFilterKeys(f, mapper); 
        
           	if ($not) { 
        
           		newFilter.$not = mapPositiveFilterKeys($not as PositiveFilter, mapper); 
        
           	} 
        
           	return newFilter;

View Diff

Sweep has identified a redundant function: The new function mapPositiveFilterKeys is redundant because its functionality is already covered by the existing mapFilterKeys function, which calls mapPositiveFilterKeys internally.

blitz-orm/src/stateMachine/query/tql/build.ts

Lines 318 to 324 in 1f80eb4

    
           const mapPositiveFilterKeys = (filter: PositiveFilter, mapper: Record<string, string>) => { 
        
           	const newFilter: PositiveFilter = {}; 
        
           	Object.entries(filter).forEach(([key, filterValue]) => { 
        
           		const newKey = mapper[key] || key; 
        
           		newFilter[newKey] = filterValue; 
        
           	}); 
        
           	return newFilter;

View Diff

src/types/requests/queries.ts

The removal of the $filterProcessed property from the EnrichedRoleQuery type may break existing logic that relies on this property to determine the processing state of a filter.

blitz-orm/src/types/requests/queries.ts

Lines 118 to 120 in 1f80eb4

    
           $filterByUnique: boolean; 
        
           $playedBy: PlayedBy; 
        
           $sort?: Sorter[];

View Diff

tests/bench.sh

The script does not handle potential errors from Docker commands, which could lead to silent failures or incomplete setups.

blitz-orm/tests/bench.sh

Lines 6 to 24 in 1f80eb4

    
           docker run --detach --rm --pull always -v $(pwd)/tests:/tests -p 8000:8000 --name $CONTAINER_NAME surrealdb/surrealdb:latest start --allow-all -u tester -p tester 
        
           until [ "`docker inspect -f {{.State.Running}} $CONTAINER_NAME`"=="true" ]; do 
        
               sleep 0.1; 
        
           done; 
        
           # Setup surrealdb database for the surrealdb test 
        
           # Create the namespace, database, and user 
        
           cat tests/surrealdb/mocks/database.surql | docker exec -i $CONTAINER_NAME ./surreal sql -u tester -p tester 
        
           # Create the schema 
        
           docker exec -it $CONTAINER_NAME ./surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000 ./tests/surrealdb/mocks/schema.surql 
        
           # Insert data 
        
           docker exec -it $CONTAINER_NAME ./surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000  ./tests/surrealdb/mocks/data.surql 
        
           # Always stop container, but exit with 1 when tests are failing 
        
           if CONTAINER_NAME=${CONTAINER_NAME} npx vitest bench $@;then 
        
               docker stop ${CONTAINER_NAME} 
        
           else 
        
               docker stop ${CONTAINER_NAME} && exit 1

View Diff

tests/surrealdb/README.md

The removal of tests/surrealdb/README.md eliminates critical setup and testing instructions, which could hinder developers from properly setting up and running tests for SurrealDB.

blitz-orm/tests/surrealdb/README.md

Lines 1 to 29 in 46a6691

    
           # SurrealDB test 
        
           Before we run test, start the db: 
        
           ```sh 
        
           surreal start file:database.db  --allow-all -u tester -p tester 
        
           ``` 
        
           Then create the database: 
        
           ```sh 
        
           cat ./tests/surrealdb/mocks/database.surql | surreal sql -u tester -p tester 
        
           ``` 
        
           Then execute the migration and seed script: 
        
           ```sh 
        
           surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000  ./tests/surrealdb/mocks/schema.surql 
        
           surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000  ./tests/surrealdb/mocks/data.surql 
        
           ``` 
        
           # Docker 
        
           ```sh 
        
           docker run --rm --pull always -v $(pwd)/tests:/tests -p 8000:8000 --name borm surrealdb/surrealdb:latest start  --allow-all -u tester -p tester 
        
           cat tests/surrealdb/mocks/database.surql | docker exec -i borm ./surreal sql -u tester -p tester 
        
           docker exec -it borm ./surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000  ./tests/surrealdb/mocks/schema.surql 
        
           docker exec -it borm ./surreal import -u tester -p tester --namespace test --database test --endpoint http://localhost:8000  ./tests/surrealdb/mocks/data.surql 
        
           ```

View Diff

tests/surrealdb/bench/all.bench.ts

The init function is used without any error handling, which could lead to unhandled exceptions if init fails.

blitz-orm/tests/surrealdb/bench/all.bench.ts

Lines 2 to 4 in 1f80eb4

    
           import { init } from '../helpers/init'; 
        
           allBench(init);

View Diff

Potential Issues

Sweep is unsure if these are issues, but they might be worth checking out.

src/stateMachine/query/bql/enrich.ts

The removal of the mapFilterKeys function means that filter keys will no longer be transformed based on the schema, which could lead to incorrect filter processing if the schema paths differ from the database paths.

blitz-orm/src/stateMachine/query/bql/enrich.ts

Lines 75 to 77 in 1f80eb4

    
           const currentSchema = getCurrentSchema(schema, node); 
        
           if (value.$filter) { 
        
           	value.$filterByUnique = checkFilterByUnique(value.$filter, currentSchema);

View Diff

The removal of the mapFilterKeys function and its associated logic means that filters will no longer be transformed based on the schema, which could lead to incorrect filter processing.

blitz-orm/src/stateMachine/query/bql/enrich.ts

Lines 75 to 77 in 1f80eb4

    
           const currentSchema = getCurrentSchema(schema, node); 
        
           if (value.$filter) { 
        
           	value.$filterByUnique = checkFilterByUnique(value.$filter, currentSchema);

View Diff

tests/typedb/bench/all.bench.ts

The init function is called within the benchmark test without any error handling, which could cause the benchmark to fail if init throws an error.

blitz-orm/tests/typedb/bench/all.bench.ts

Lines 2 to 4 in 1f80eb4

    
           import { init } from '../helpers/init'; 
        
           allBench(init);

View Diff

package.json was not reviewed because our filter identified it as typically a non-human-readable or less important file (e.g., dist files, package.json, images). If this is an error, please let us know.

lveillard · 2024-05-29T16:25:08Z

Sweep: Please fix these:

File: tests/bench.sh
The script does not handle potential errors from Docker commands, which could lead to silent failures or incomplete setups.

File: tests/surrealdb/bench/all.bench.ts
The init function is used without any error handling, which could lead to unhandled exceptions if init fails.

sweep-ai-deprecated · 2024-05-29T16:25:26Z

❌ An error occured!

The exception message is:/tmp/cache/repos/Blitzapps/blitz-orm/b4e33ac3c01cd732fc6f5e85ce06c9d85d6107d5b027f9bd249c8c36a2adcc39/feat-surrealdb-queries/tests/surrealdb/README.md does not exist.

The stack trace is:Traceback (most recent call last):
File "/app/sweepai/handlers/on_comment.py", line 277, in on_comment
snippets_modified = [Snippet.from_file(
File "/app/sweepai/handlers/on_comment.py", line 278, in
pr_diff.file_name, cloned_repo.get_file_contents(pr_diff.file_name)
File "/app/sweepai/utils/github_utils.py", line 501, in get_file_contents
raise FileNotFoundError(f"{local_path} does not exist.")
FileNotFoundError: /tmp/cache/repos/Blitzapps/blitz-orm/b4e33ac3c01cd732fc6f5e85ce06c9d85d6107d5b027f9bd249c8c36a2adcc39/feat-surrealdb-queries/tests/surrealdb/README.md does not exist.

Please report this on our community forum.

This is an automated message generated by Sweep AI.

lveillard · 2024-05-29T16:40:17Z

Sweep: Please fix these:

File: tests/bench.sh
The script does not handle potential errors from Docker commands, which could lead to silent failures or incomplete setups.

File: tests/surrealdb/bench/all.bench.ts
The init function is used without any error handling, which could lead to unhandled exceptions if init fails.

Dont explore this file, as it was deleted in this PR, so please dont review it:
/tests/surrealdb/README.md

sweep-ai-deprecated · 2024-05-29T16:40:36Z

❌ An error occured!

The exception message is:/tmp/cache/repos/Blitzapps/blitz-orm/617a5e3467d60497fb4105c52b112ca8c9cae585007af96beb9801214aaf4f74/feat-surrealdb-queries/tests/surrealdb/README.md does not exist.

The stack trace is:Traceback (most recent call last):
File "/app/sweepai/handlers/on_comment.py", line 277, in on_comment
snippets_modified = [Snippet.from_file(
File "/app/sweepai/handlers/on_comment.py", line 278, in
pr_diff.file_name, cloned_repo.get_file_contents(pr_diff.file_name)
File "/app/sweepai/utils/github_utils.py", line 501, in get_file_contents
raise FileNotFoundError(f"{local_path} does not exist.")
FileNotFoundError: /tmp/cache/repos/Blitzapps/blitz-orm/617a5e3467d60497fb4105c52b112ca8c9cae585007af96beb9801214aaf4f74/feat-surrealdb-queries/tests/surrealdb/README.md does not exist.

Please report this on our community forum.

This is an automated message generated by Sweep AI.

feat:surrealDB queries 39-23

1f80eb4

lveillard added 7 commits May 24, 2024 13:09

feat:SurrealDB queries 44-18

2c79fd1

chore:renaming and small fixes 44-18

d1002ca

fix: some minor fixes and generated schema tests 46-16

fca0273

fix: added inheritance mark on fields 45-17

c39e5d9

fix: simplified enriched metadata for subtypes 46-16

52a167a

feat: sort, refacto, split enriched metadata by db 48-14

cafac17

feat: most surrealDB queries working

3a97021

lveillard added the sweep label May 29, 2024

lveillard mentioned this pull request May 29, 2024

Feature: Bidirectional link records surrealdb/surrealdb#4118

Closed

2 tasks

feat: schema simplification

9340d21

lveillard marked this pull request as ready for review May 31, 2024 14:28

lveillard merged commit 764f1bb into main May 31, 2024

lveillard deleted the feat-surrealdb-queries branch May 31, 2024 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat:surrealDB queries 39-23 #66

feat:surrealDB queries 39-23 #66

Uh oh!

lveillard commented May 23, 2024

Uh oh!

sweep-ai-deprecated bot commented May 23, 2024 •

edited

Loading

Sweep Found These Issues

Potential Issues

Uh oh!

lveillard commented May 29, 2024

Uh oh!

sweep-ai-deprecated bot commented May 29, 2024 •

edited

Loading

Uh oh!

lveillard commented May 29, 2024

Uh oh!

sweep-ai-deprecated bot commented May 29, 2024 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

feat:surrealDB queries 39-23 #66

feat:surrealDB queries 39-23 #66

Uh oh!

Conversation

lveillard commented May 23, 2024

Uh oh!

sweep-ai-deprecated bot commented May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sweep: PR Review

Sweep Found These Issues

Potential Issues

Uh oh!

lveillard commented May 29, 2024

Uh oh!

sweep-ai-deprecated bot commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lveillard commented May 29, 2024

Uh oh!

sweep-ai-deprecated bot commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sweep-ai-deprecated bot commented May 23, 2024 •

edited

Loading

sweep-ai-deprecated bot commented May 29, 2024 •

edited

Loading

sweep-ai-deprecated bot commented May 29, 2024 •

edited

Loading