[IcebergIO] Add Iceberg SQL table provider and tests #34799
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
| + "LOCATION '" | ||
| + tableIdentifier | ||
| + "' \n" | ||
| + "TBLPROPERTIES '{ \"triggering_frequency_seconds\" : 10 }'"; |
There was a problem hiding this comment.
Can I define partition column with transform function for my Iceberg Table ?
There was a problem hiding this comment.
We don't support creating tables with partition spec right now (tracker #34117)
| + " c_timestamp TIMESTAMP, \n" | ||
| + " c_varchar VARCHAR, \n " | ||
| + " c_char CHAR, \n" | ||
| + " c_arr ARRAY<VARCHAR> \n" |
There was a problem hiding this comment.
Could you also add a nested array field such as array<array> and some nested types also ?
There was a problem hiding this comment.
Thanks for suggesting this, it pointed out some incorrect conversion logic: #34887
There was a problem hiding this comment.
Fixed conversion logic and push a new commit
(lmk if you'd like me to separate it into a different PR)
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #34799 +/- ##
============================================
- Coverage 54.52% 54.50% -0.03%
- Complexity 1479 1491 +12
============================================
Files 1010 1012 +2
Lines 160465 160407 -58
Branches 1079 1086 +7
============================================
- Hits 87499 87427 -72
- Misses 70861 70873 +12
- Partials 2105 2107 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Assigning reviewers: R: @robertwb for label java. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Integrate IcebergIO with Beam SQL