-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Describe the bug
When using either SchemaDefinition.<T>builder().withPojo()
(or directly Schema.AVRO()
) time conversions for logical types are registered so POJOs with some java.time objects (not all) are correctly serialized.
However when method SchemaDefinition.<T>builder().withJsonDef()
is used instead such registration is not performed. We have to manually register those types for these conversions to work:
import org.apache.pulsar.shade.org.apache.avro.data.TimeConversions;
import org.apache.pulsar.shade.org.apache.avro.generic.GenericData;
import org.apache.pulsar.shade.org.apache.avro.reflect.ReflectData;
private static final GenericData[] GD_INSTANCES = { ReflectData.AllowNull.get(), ReflectData.get(), GenericData.get() };
// Static registration using shaded objects before reading any schema
static {
for (GenericData gd : GD_INSTANCES) {
gd.addLogicalTypeConversion(new TimeConversions.DateConversion());
gd.addLogicalTypeConversion(new TimeConversions.TimeMillisConversion());
gd.addLogicalTypeConversion(new TimeConversions.TimestampMillisConversion());
}
}
Weirdly enough we have found that using a POJO with a java.time.Instant
and using an schema with a LogicalType timestamp-millis
it is converted to Long
and assigned to field in POJO that has type java.lang.Instant
(How can this even be possible?).
To Reproduce
Create a POJO with an Instant field.
Produce to a topic using that POJO (and registering the Schema).
On the consumer side use the AVRO schema text with SchemaDefinition.<T>builder().withPojo()
to read from the topic. Review what is contained in the read objetcts.
For example. If POJO type is TestPojo
we could use:
Schema<TestPojo> schema = AvroSchema.of(SchemaDefinition.<TestPojo>builder().withJsonDef(org.apache.avro.reflect.ReflectData.AllowNull.get().getSchema(TestPojo.class)).build());
And use that Schema
to create a consumer.
Expected behavior
That same logical conversions are applied both using withJsonDef()
or with withPojo()
Screenshots
Not applicable
Desktop (please complete the following information):
Windows
Additional context
This related to #15858