a way to improve encoding performance for messages batches when using AvroTurf::Messaging #206
Description
Hello there,
We have this case where we are encoding over 1000 messages at a time, but I noticed a bottleneck where AvroTurf takes over 60% of the CPU time, so I profiled the code and found out that the schema is being parsed for every message as noticed in below code, is this intentional? is there anyway to improve it ?
note: even in fetch_schema_by_id the method @schemas_by_id is not being set anywhere.
appreciate any help thank you :)
# Providing subject and version to determine the schema,
# which skips the auto registeration of schema on the schema registry.
# Fetch the schema from registry with the provided subject name and version.
def fetch_schema(subject:, version: 'latest')
schema_data = @registry.subject_version(subject, version)
schema_id = schema_data.fetch('id')
schema = Avro::Schema.parse(schema_data.fetch('schema'))
[schema, schema_id]
end
# Fetch the schema from registry with the provided schema_id.
def fetch_schema_by_id(schema_id)
schema = @schemas_by_id.fetch(schema_id) do
schema_json = @registry.fetch(schema_id)
Avro::Schema.parse(schema_json)
end
[schema, schema_id]
end