The documentation repository for the Karafka ecosystem - a Ruby and Rails efficient Kafka processing framework.

What is this repository?

This repository contains all the documentation for Karafka and its related components, organized as a collection of Markdown files that are automatically processed and deployed

Automated Content Management

This repository includes several automated processes to keep documentation current and consistent:

Automatic Content Refresh

The repository automatically fetches and updates dynamic content twice daily (6:00 AM and 6:00 PM UTC) and on-demand through repository dispatch events:

• Component Changelogs: Automatically pulls the latest CHANGELOG.md files from all Karafka ecosystem repositories
• librdkafka Releases: Fetches and formats librdkafka release information from GitHub
• Error Documentation: Generates comprehensive librdkafka error reference documentation

Content Processing Scripts

Located in the bin/ directory:

• refresh_changelogs - Downloads changelogs from component repositories
• refresh_librdkafka_errors - Generates librdkafka error documentation dynamically using the rdkafka gem
• refresh_librdkafka_releases - Generates librdkafka releases documentation dynamically based on the official releases
• refresh_librdkafka_statistics - Generates librdkafka statistics documentation based on the official docs
• refresh_librdkafka_configuration - Generates librdkafka configuration documentation based on the official docs
• align_structure - Flattens nested documentation structure and fixes cross-references for MkDocs compatibility
• mklint - Validates documentation structure, references, and builds using MkDocs in strict mode
• sync_gh - Synchronizes content with GitHub Karafka main repo

Continuous Integration

The CI pipeline ensures documentation quality through:

• Markdown Linting: Uses markdownlint-cli2 to enforce consistent Markdown formatting
• Structure Validation: Validates cross-references, anchor links, and overall documentation structure
• Security: Verifies all GitHub Actions use SHA-pinned versions for security

Pull Request Automation

When automatic content updates are available, the system:

1. Fetches the latest content from all sources
2. Regenerates dynamic documentation
3. Creates a pull request with the changes
4. Maintains a clean commit history with automated updates

Auto-Generated Content

Several files in this repository are automatically generated and should not be edited manually:

• Changelog/*.md - Component changelogs
• Librdkafka/Changelog.md - librdkafka release notes
• Librdkafka/Errors.md - librdkafka error reference

These files include header comments indicating their auto-generated status:

[//]: # (This file is auto-generated by bin/refresh_remote_content)
[//]: # (Do not edit manually - changes will be overwritten)

License

See LICENSE for details.