Unexpected drift reported for message.timestamp.before.max.ms, message.timestamp.after.max.ms settings #191
Description
When running topicctl check, we're getting drift-related failures for two settings on a particular topic:
[2024-04-17 09:47:27] INFO Check failed for topic <topic> (cluster=<cluster>, env=staging):
-----------------------------+----+----------------------------------------------------------------------------------------------------------------------------------
NAME | OK | DETAILS
-----------------------------+----+----------------------------------------------------------------------------------------------------------------------------------
config correct | ✓ |
configs consistent | ✓ |
topic exists | ✓ |
config settings correct | ✗ | 2 keys have different values between cluster and topic config: [message.timestamp.after.max.ms message.timestamp.before.max.ms]
replication factor correct | ✓ |
partition count correct | ✓ |
throttles clear | ✓ |
replicas in-sync | ✓ |
Our topic configuration YAML does not explicitly configure these settings. From what I can tell, these happen to be fairly new settings introduced in Kafka 3.6 (Oct 2023). I noticed a similar issue in the Confluent Terraform provider. Perhaps these settings need to be added to topicctl?
Additional details:
confluent kafka topic describe reports the following values for these settings (these are the defaults):
message.timestamp.after.max.ms 9223372036854775807
message.timestamp.before.max.ms 9223372036854775807
When running apply, I see the following output:
[2024-04-17 10:32:52] WARN Found 2 key(s) set in cluster but missing from config:
----------------------------------+----------------------
KEY | CLUSTER VALUE
----------------------------------+----------------------
message.timestamp.before.max.ms | 9223372036854775807
message.timestamp.after.max.ms | 9223372036854775807
----------------------------------+----------------------
These will be left as-is.
Running check after the apply results in the same output as above.
topicctlversion: v1.16.1- Kafka version: 3.6 (Confluent Cloud)