Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add List of nested types support for Arrow vtab: Struct, List, FixedSizeList #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Feb 11, 2025
Merged

Add List of nested types support for Arrow vtab: Struct, List, FixedSizeList #432

merged 2 commits into from
Feb 11, 2025

Conversation

@sgrebnov
Copy link
Contributor

@sgrebnov sgrebnov commented Feb 11, 2025 •
edited
Loading

Add List of nested types (List , Struct, FixedSizeList) support for Arrow vtab.

Record batch from let rb = stmt.query_arrow(param)?.next().expect("no record batch"); to demonstrate the data used in the tests added

---- vtab::arrow::test::test_list_of_fixed_size_lists_roundtrip stdout ----
+--------------------+
| a                  |
+--------------------+
| [[1, 2], , [3, 4]] |
|                    |
+--------------------+
---- vtab::arrow::test::test_list_of_structs_roundtrip stdout ----
+----------------------------------+
| a                                |
+----------------------------------+
| [{i: 1, s: foo}, {i: 2, s: baz}] |
|                                  |
| [{i: 5, s: baz}]                 |
+----------------------------------+
---- vtab::arrow::test::test_list_of_lists_roundtrip stdout ----
+----------------------+
| a                    |
+----------------------+
| [[1, 2], , [3, , 5]] |
|                      |
+----------------------+

@sgrebnov
Copy link
Contributor Author

The improvement has been actively used as part of the https://github.com/spiceai/spiceai and https://github.com/datafusion-contrib/datafusion-table-providers projects since Dec 22. I had missed sending the upstream PR earlier, so I am submitting it now.

@sgrebnov sgrebnov mentioned this pull request Feb 11, 2025
Closed
@Maxxen
Copy link
Member

Maxxen commented Feb 11, 2025

LGTM, thanks!

@Maxxen Maxxen merged commit e38faee into duckdb:main Feb 11, 2025
3 checks passed
@sgrebnov
Copy link
Contributor Author

Thank you @Maxxen!

@lukekim lukekim mentioned this pull request Feb 15, 2025
Closed
sgrebnov added a commit to spiceai/duckdb-rs that referenced this pull request Feb 23, 2025
@sgrebnov
Add List of nested types support for Arrow vtab: Struct, List, FixedS…
02757a8 
…izeList (duckdb#432)

* Add List of nested types support for Arrow vtab: Struct, List, FixedSizeList

* Fix lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sgrebnov @Maxxen