-
Notifications
You must be signed in to change notification settings - Fork 3
Traefik
This guide provides recommended Traefik configurations optimized for LogLynx log ingestion and analysis.
- Quick Start
- JSON Format Configuration (Recommended)
- CLF Format Configuration
- Access Log Settings
- Performance Tuning
- Troubleshooting
For immediate setup with LogLynx, add this to your Traefik configuration:
version: '3.8'
services:
traefik:
image: traefik:v2.10
command:
- "--accesslog=true"
- "--accesslog.filepath=/logs/access.log"
- "--accesslog.format=json"
- "--accesslog.bufferingsize=100"
volumes:
- ./logs:/logs
- /var/run/docker.sock:/var/run/docker.sock:roaccessLog:
filePath: "/var/log/traefik/access.log"
format: json
bufferingSize: 100LogLynx works best with Traefik's JSON format, as it captures all available fields with maximum precision.
# traefik.yml (static configuration)
accessLog:
# File path
filePath: "/var/log/traefik/access.log"
# Format: json is highly recommended for LogLynx
format: json
# Buffer size (number of log lines to buffer before writing)
# Higher values = better performance, but potential data loss on crash
# Recommended: 100-1000 for production
bufferingSize: 100
# Fields to log (optional - logs all fields by default)
fields:
defaultMode: keep
# Additional headers to capture (optional)
headers:
defaultMode: keep
names:
User-Agent: keep
Referer: keep
X-Forwarded-For: keep
X-Forwarded-Proto: keep
X-Real-Ip: keepWhen using JSON format, LogLynx captures:
Client Information:
-
ClientAddr→ ClientIP + ClientPort -
ClientHost→ ClientHostname -
ClientUsername→ ClientUser
Request Information:
-
RequestMethod→ Method -
RequestProtocol→ Protocol -
RequestHost→ Host -
RequestPath→ Path -
RequestContentSize→ RequestLength -
request_X-Forwarded-Proto→ RequestScheme
Response Information:
-
DownstreamStatus→ StatusCode -
DownstreamContentSize→ ResponseSize -
downstream_Content-Type→ ResponseContentType
Timing Information:
-
Duration→ Duration (nanoseconds) -
StartUTC→ StartUTC (nanosecond precision timestamp) -
RequestTime→ ResponseTimeMs
Proxy/Backend Information:
-
ServiceName→ BackendName -
ServiceURL→ BackendURL -
RouterName→ RouterName -
OriginStatus→ UpstreamStatus -
OriginDuration→ UpstreamResponseTimeMs -
origin_Content-Type→ UpstreamContentType -
RetryAttempts→ RetryAttempts
{
"ClientAddr": "103.4.250.66:48952",
"ClientHost": "103.4.250.66",
"ClientPort": "48952",
"ClientUsername": "-",
"DownstreamContentSize": 20,
"DownstreamStatus": 200,
"Duration": 8528869,
"OriginDuration": 8000000,
"OriginStatus": 200,
"Overhead": 528869,
"RequestAddr": "example.com",
"RequestContentSize": 0,
"RequestCount": 49384,
"RequestHost": "example.com",
"RequestMethod": "GET",
"RequestPath": "/_app/version.json",
"RequestPort": "-",
"RequestProtocol": "HTTP/2.0",
"RequestScheme": "https",
"RetryAttempts": 0,
"RouterName": "llm@file",
"ServiceAddr": "192.168.241.152:8181",
"ServiceName": "llm@file",
"ServiceURL": "http://192.168.241.152:8181/",
"StartLocal": "2025-11-06T22:06:44.123456789+00:00",
"StartUTC": "2025-11-06T22:06:44.123456789Z",
"downstream_Content-Type": "application/json",
"origin_Content-Type": "application/json",
"request_User-Agent": "Mozilla/5.0...",
"request_X-Forwarded-For": "103.4.250.66",
"request_X-Forwarded-Proto": "https"
}If you prefer Common Log Format (CLF), LogLynx supports it but with reduced field capture.
# traefik.yml
accessLog:
filePath: "/var/log/traefik/access.log"
format: common # CLF format
bufferingSize: 100192.168.241.1 - - [06/Nov/2025:22:06:44 +0000] "GET /_app/version.json HTTP/2.0" 200 20 "-" "-" 49384 "llm@file" "http://192.168.241.152:8181/" 9ms
| CLF Position | Field | LogLynx Field |
|---|---|---|
| 1 | Client IP | ClientIP |
| 2 | Ident | (not used) |
| 3 | Username | ClientUser |
| 4 | Timestamp | Timestamp |
| 5 | Method + Path + Protocol | Method, Path, Protocol |
| 6 | Status Code | StatusCode |
| 7 | Response Size | ResponseSize |
| 8 | Referer | Referer |
| 9 | User-Agent | UserAgent |
| 10 | Request Count | (not used) |
| 11 | Router Name | RouterName |
| 12 | Service URL | BackendURL |
| 13 | Duration | Duration (converted ms→ns) |
Note: CLF format does not capture:
- StartUTC (constructed from timestamp, second precision only)
- RetryAttempts
- UpstreamStatus
- Content-Type headers
- Client hostname
- Request scheme
- Nanosecond precision timing
Recommendation: Use JSON format for complete field capture.
Docker/Container Environments:
accessLog:
filePath: "/logs/access.log"Mount /logs volume to host for LogLynx ingestion.
Bare Metal/VM:
accessLog:
filePath: "/var/log/traefik/access.log"Ensure LogLynx has read permissions.
Kubernetes:
accessLog:
filePath: "/dev/stdout" # Send to stdout for log aggregatorUse log shipper (Fluentd/Fluent Bit) to forward to LogLynx.
accessLog:
bufferingSize: 100 # Number of log lines to bufferBuffer Size Guidelines:
| Environment | Buffer Size | Reason |
|---|---|---|
| Development | 10-50 | Fast visibility, frequent writes |
| Production (Low Traffic) | 100-500 | Balance performance/safety |
| Production (High Traffic) | 500-1000 | Maximum performance |
| Mission Critical | 0 (disabled) | Immediate writes, no loss risk |
Trade-offs:
- Higher buffer = Better performance, potential data loss on crash
- Lower buffer = More disk I/O, safer against crashes
- No buffer (0) = Real-time writes, highest I/O overhead
Exclude health checks or internal traffic:
accessLog:
filePath: "/var/log/traefik/access.log"
format: json
filters:
statusCodes:
- "200-299" # Only successful requests
retryAttempts: true # Only requests with retries
minDuration: "10ms" # Only slow requestsWarning: LogLynx performs its own filtering. Using Traefik filters may limit analysis capabilities.
For environments with >1000 req/s:
accessLog:
filePath: "/var/log/traefik/access.log"
format: json
bufferingSize: 1000 # Large buffer
# Optional: Reduce logged fields
fields:
defaultMode: keep
headers:
defaultMode: drop # Don't log all headers
names:
User-Agent: keep
Referer: keepMount log volume on fast storage:
volumes:
- type: bind
source: /mnt/ssd/traefik-logs # SSD storage
target: /logsOr use tmpfs (RAM) for extreme performance:
volumes:
- type: tmpfs
target: /logs
tmpfs:
size: 1G # 1GB RAM diskWarning: tmpfs logs are lost on restart. Use log shipper to forward immediately.
Configure LogLynx source for optimal performance:
# LogLynx config.yaml
sources:
- name: "traefik-prod"
type: "traefik"
path: "/var/log/traefik/access.log"
format: "json"
# Polling interval (how often to check for new logs)
poll_interval: "1s" # Default
# Batch size (how many log lines to process at once)
batch_size: 1000 # Default
# Buffer size (internal LogLynx buffer)
buffer_size: 10000 # DefaultPerformance Guidelines:
| Traffic Level | poll_interval | batch_size | buffer_size |
|---|---|---|---|
| Low (<100 req/s) | 5s | 500 | 5000 |
| Medium (100-500 req/s) | 2s | 1000 | 10000 |
| High (500-2000 req/s) | 1s | 2000 | 20000 |
| Very High (>2000 req/s) | 500ms | 5000 | 50000 |
Symptoms: LogLynx shows no new logs, but Traefik is receiving traffic
Checks:
-
Verify log file exists:
ls -lh /var/log/traefik/access.log
-
Check file permissions:
# LogLynx needs read permission chmod 644 /var/log/traefik/access.log -
Verify log format:
head -n 1 /var/log/traefik/access.log
Should see JSON if
format: jsonis configured. -
Check Traefik is writing logs:
tail -f /var/log/traefik/access.log
Send a test request and verify new log line appears.
-
Verify LogLynx source configuration:
sources: - name: "traefik-prod" type: "traefik" path: "/var/log/traefik/access.log" # Must match actual path format: "json" # Must match Traefik config
Symptoms: LogLynx or Traefik consuming excessive CPU
Solutions:
-
Increase buffer size (Traefik writes less frequently):
accessLog: bufferingSize: 1000 # Increase from 100
-
Reduce LogLynx polling frequency:
sources: - name: "traefik-prod" poll_interval: "5s" # Increase from 1s
-
Use batch processing:
sources: - name: "traefik-prod" batch_size: 5000 # Increase from 1000
Symptoms: Disk at 100% utilization, slow log processing
Solutions:
-
Enable log compression (logrotate):
/var/log/traefik/access.log { compress delaycompress } -
Use faster storage (SSD instead of HDD)
-
Reduce logged fields:
accessLog: fields: headers: defaultMode: drop # Don't log all headers
-
Use tmpfs for temporary log storage:
volumes: - type: tmpfs target: /logs tmpfs: size: 1G
Symptoms: Some fields show empty in LogLynx dashboard
Checks:
-
Verify JSON format (not CLF):
accessLog: format: json # Not "common"
-
Check Traefik version (older versions may not have all fields):
traefik version # Recommended: v2.10 or later -
Verify field logging is enabled:
accessLog: fields: defaultMode: keep # Not "drop"
-
Check raw log contains field:
cat /var/log/traefik/access.log | jq '.[0]'
Symptoms: Disk space filling up, large log files
Solutions:
-
Enable log rotation:
/var/log/traefik/access.log { daily rotate 30 size 100M } -
Reduce retention period:
rotate 7 # Keep only 7 days instead of 30 -
Use filtering (Traefik side):
accessLog: filters: statusCodes: - "400-599" # Only log errors
-
Archive old logs:
# Compress logs older than 7 days find /var/log/traefik/ -name "*.log" -mtime +7 -exec gzip {} \;
Symptoms: Same request appears multiple times in LogLynx
Causes & Solutions:
-
Multiple LogLynx instances reading same log:
- Use single instance
- Or configure deduplication properly
-
Log file was re-imported:
- LogLynx's deduplication system should handle this
- Check hash is calculated correctly (9 fields)
-
Hash collision (extremely rare):
- Check if requests are actually different
- SHA256 collision probability: ~0% for practical purposes
- ✅ Use JSON format for maximum field capture
- ✅ Enable buffering (100-1000) for better performance
- ✅ Set up log rotation to manage disk space
- ✅ Use SSD storage for log files in high-traffic environments
- ✅ Monitor disk space and I/O utilization
- ✅ Test configuration with sample traffic before production
- ✅ Set correct file permissions (644 for log files)
- ✅ Use log shipping in distributed deployments
- ❌ Don't use CLF format unless necessary (fewer fields)
- ❌ Don't disable buffering in high-traffic environments (performance impact)
- ❌ Don't filter logs at Traefik level (limits LogLynx analysis)
- ❌ Don't use HDD storage for high-traffic logs (I/O bottleneck)
- ❌ Don't forget log rotation (disk will fill up)
- ❌ Don't mix JSON and CLF formats in same file
- ❌ Don't manually edit log files (corrupts LogLynx tracking)
- ❌ Don't use world-writable permissions (security risk)
- Traefik Access Logs Documentation: https://doc.traefik.io/traefik/observability/access-logs/
- Log Rotation: https://linux.die.net/man/8/logrotate
- Fluent Bit: https://docs.fluentbit.io/manual/
Last Updated: November 2025
- Home - Introduction and overview
- API Documentation - Complete API reference
- Environment Variables - Configuration guide
- Standalone Deployment - Native installation
- Docker Deployment - Container deployment