[exporter/kafka] franz-go library fails to connect to AWS MSK using IAM #42781
Description
Component(s)
exporter/kafka
What happened?
Description
After updating to 0.135.0, the new library is failing to connect using SASL together with AWS IAM OAUTHBEARER token.
Steps to Reproduce
Update image to 0.135.0 with the new franz-go feature gate enabled.
Expected Result
No errors
Actual Result
Got error:
invalid large response size 352518912 > limit 104857600; the first three bytes received appear to be a tls alert record for TLS v1.2; is this a plaintext connection speaking to a tls endpoint?
Collector version
0.135.0
Environment information
Environment
OS: Ubuntu 22.04
OpenTelemetry Collector configuration
receivers:
journald/os:
directory: /var/log/journal
priority: info
processors:
batch: {}
memory_limiter:
check_interval: 1s
limit_mib: 500
spike_limit_mib: 100
exporters:
kafka/os:
brokers: ${env:AWS_MSK_LOGGING_SERVERS}
producer:
compression: gzip
client_id: otel-collector-ec2
topic: logging.syslog.v1.ingestion
protocol_version: 3.5.2
encoding: raw
auth:
sasl:
mechanism: "AWS_MSK_IAM_OAUTHBEARER"
aws_msk:
region: ${env:AWS_REGION}
service:
pipelines:
logs/os:
receivers:
- journald/os
processors:
- batch
- memory_limiter
exporters:
- kafka/osLog output
2025-09-19T07:37:07.738Z info franz [email protected]/kzap.go:114 immediate metadata update triggered {"resource": {"service.instance.id": "0667001d-fb15-4748-a1f6-0839bb4d6011", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka/os", "otelcol.component.kind": "exporter", "otelcol.signal": "logs", "why": "forced load because we are producing to a topic for the first time"}
2025-09-19T07:37:07.739Z info franz [email protected]/kzap.go:114 producing to a new topic for the first time, fetching metadata to learn its partitions {"resource": {"service.instance.id": "0667001d-fb15-4748-a1f6-0839bb4d6011", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka/os", "otelcol.component.kind": "exporter", "otelcol.signal": "logs", "topic": "logging.syslog.v1.ingestion"}
2025-09-19T07:37:07.749Z error franz [email protected]/kzap.go:112 unable to request api versions {"resource": {"service.instance.id": "0667001d-fb15-4748-a1f6-0839bb4d6011", "service.name": "otelcol-contrib", "service.version": "0.135.0"}, "otelcol.component.id": "kafka/os", "otelcol.component.kind": "exporter", "otelcol.signal": "logs", "broker": "seed_0", "err": "invalid large response size 352518912 > limit 104857600; the first three bytes received appear to be a tls alert record for TLS v1.2; is this a plaintext connection speaking to a tls endpoint?"}
github.com/twmb/franz-go/plugin/kzap.(*Logger).Log
github.com/twmb/franz-go/plugin/[email protected]/kzap.go:112
github.com/twmb/franz-go/pkg/kgo.(*wrappedLogger).Log
github.com/twmb/[email protected]/pkg/kgo/logger.go:125
github.com/twmb/franz-go/pkg/kgo.(*brokerCxn).init
github.com/twmb/[email protected]/pkg/kgo/broker.go:741
github.com/twmb/franz-go/pkg/kgo.(*broker).loadConnection
github.com/twmb/[email protected]/pkg/kgo/broker.go:578
github.com/twmb/franz-go/pkg/kgo.(*broker).handleReq
github.com/twmb/[email protected]/pkg/kgo/broker.go:302
github.com/twmb/franz-go/pkg/kgo.(*broker).handleReqs
github.com/twmb/[email protected]/pkg/kgo/broker.go:286Additional context
No response
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.