Follow up to address comments in variant array writer #12933
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rdblue
reviewed
Apr 30, 2025
| return list(shredType); | ||
| } | ||
|
|
||
| return null; |
There was a problem hiding this comment.
Do you think there are cases where we want to shred an array of mixed types, with each value stored in value as an encoded variant? I'm not sure.
There was a problem hiding this comment.
I don't think we will have the case that all the elements will be stored in value since I would expect engines will at least choose a shredding type that matches some of the elements. I think this is more common: by sampling the elements, we choose INT32 as the shredding type while some are strings. If all of theme are in value and they are binary, that doesn't seem to be useful.
rdblue
approved these changes
Apr 30, 2025
|
Thanks, @aihuaxu! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix up to address the comments in #12847: update the shredding to only apply when all the array elements are of a uniform type instead of using most common type. That will make verification straightforward.