Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Build: Bump Parquet-Java to 1.16.0 #13941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Sep 3, 2025
Merged

Build: Bump Parquet-Java to 1.16.0 #13941

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9d97155
Parquet: Test out the Parquet-Java 1.16.0 release
Fokko Aug 28, 2025
4d005ad
Merge branch 'main' of github.com:apache/iceberg into fd-test-parquet…
Fokko Aug 31, 2025
5e6d668
Merge branch 'main' of github.com:apache/iceberg into fd-test-parquet…
Fokko Sep 2, 2025
6309f60
:/
Fokko Sep 2, 2025
5df9a4b
Set the annotation
Fokko Sep 2, 2025
871040c
Remove staging
Fokko Sep 2, 2025
96fddd4
Introduce constant for the Variant version
Fokko Sep 2, 2025
da30312
Merge branch 'main' of github.com:apache/iceberg into fd-test-parquet…
Fokko Sep 2, 2025
a2558cc
Merge branch 'fd-test-parquet-1-16' of github.com:Fokko/iceberg into …
Fokko Sep 2, 2025
7a2bf7d
Allow for both
Fokko Sep 2, 2025
8259b45
Revert relying on Iceberg schema
Fokko Sep 2, 2025
fa18217
Improve error message
Fokko Sep 2, 2025
3b4ba3e
Add sType instanceof VariantType
Fokko Sep 2, 2025
2e77e6c
trigger ci
kevinjqliu Sep 3, 2025
a86fe2a
Merge branch 'main' of github.com:apache/iceberg into fd-test-parquet…
Fokko Sep 3, 2025
d6edcb1
Thanks Aihua
Fokko Sep 3, 2025
bcf6fd4
Merge branch 'fd-test-parquet-1-16' of github.com:Fokko/iceberg into …
Fokko Sep 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Thanks Aihua
  • Loading branch information
@Fokko
Fokko committed Sep 3, 2025
commit d6edcb1cde6be4be294d5f8005d78944f24f9f68
3 changes: 2 additions & 1 deletion parquet/src/main/java/org/apache/iceberg/parquet/TypeWithSchemaVisitor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ public static <T> T visit(
return visitList(iType, group, visitor);
} else if (annotation instanceof MapLogicalTypeAnnotation) {
return visitMap(iType, group, visitor);
} else if (iType != null && iType.isVariantType()) {
} else if (annotation instanceof LogicalTypeAnnotation.VariantLogicalTypeAnnotation
|| (iType != null && iType.isVariantType())) {
// when Parquet has a VARIANT logical type, use it here
return visitVariant(iType.asVariantType(), group, visitor);
}
Expand Down
7 changes: 5 additions & 2 deletions ...v4.0/spark/src/main/java/org/apache/iceberg/spark/data/ParquetWithSparkSchemaVisitor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,8 +155,11 @@ public static <T> T visit(DataType sType, Type type, ParquetWithSparkSchemaVisit
} finally {
visitor.fieldNames.pop();
}
} else if (sType instanceof VariantType
|| LogicalTypeAnnotation.variantType(Variant.VARIANT_SPEC_VERSION).equals(annotation)) {
} else if (LogicalTypeAnnotation.variantType(Variant.VARIANT_SPEC_VERSION).equals(annotation)
|| sType instanceof VariantType) {
// For the Variant we both check the Parquet LogicalTypeAnnotation, and we rely on the
// Iceberg schema, since there are engines like Spark that produce VariantTypes without the
// annotation.
Preconditions.checkArgument(
sType instanceof VariantType,
"Invalid variant: Spark type %s is not a variant type",
Expand Down