update to latest kafka-manager (kafka.common.KafkaException: Unknown offset schema version 3 with kafka 2.1.1) #257
Description
Using kafka version 2.1.1 I see a lot of warn in the kafka-manager after activating it with:
kubectl apply -f yahoo-kafka-manager/
The warnings are:
[warn] k.m.a.c.KafkaManagedOffsetCache - Failed to process a message from offset topic on cluster microk8s!
kafka.common.KafkaException: Unknown offset schema version 3
at kafka.manager.utils.one10.GroupMetadataManager$.schemaForOffset(GroupMetadataManager.scala:428) ~[kafka-manager.kafka-manager-1.3.3.18-sans-externalized.jar:na]
at kafka.manager.utils.one10.GroupMetadataManager$.readOffsetMessageValue(GroupMetadataManager.scala:532) ~[kafka-manager.kafka-manager-1.3.3.18-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaManagedOffsetCache$$anonfun$run$4.apply(KafkaStateActor.scala:332) [kafka-manager.kafka-manager-1.3.3.18-sans-externalized.jar:na]
at kafka.manager.actor.cluster.KafkaManagedOffsetCache$$anonfun$run$4.apply(KafkaStateActor.scala:308) [kafka-manager.kafka-manager-1.3.3.18-sans-externalized.jar:na]
at scala.util.Success.foreach(Try.scala:236) [org.scala-lang.scala-library-2.11.12.jar:na]
at kafka.manager.actor.cluster.KafkaManagedOffsetCache.run(KafkaStateActor.scala:308) [kafka-manager.kafka-manager-1.3.3.18-sans-externalized.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]
I think we need the latest version from https://github.com/yahoo/kafka-manager (or at least a version which contains changes from yahoo/CMAK#620). Unfortunately I could not find a pre-built image: solsson/kafka-manager image. I did try to build it myself but got an error. @solsson: do you plan to update to your latest kafka-manager version?
I see that the latest kafka-manager would also support the latest kafka version (2.2.0), see yahoo/CMAK#626