HTTP Dispatcher is a high-performance HTTP request processing and forwarding server written in Rust. It supports data transformation using VRL (Vector Remap Language) and provides flexible configuration options for processing and forwarding HTTP requests.

• 🚀 High-performance HTTP request processing
• 🔄 Data transformation using VRL (Vector Remap Language)
• ⚙️ Flexible YAML-based configuration
• 🔌 Multiple target forwarding support
• 🔄 Built-in retry mechanism with configurable attempts
• 🔍 Built-in /echo endpoint for debugging
• 📝 Request templating with variable substitution
• 📊 Detailed logging with structured output
• 🔑 Custom HTTP headers and query parameters support
• 🛡️ Configurable request size limits
• ⚡ Async processing with Tokio runtime

• Rust 1.70 or higher
• Supported operating systems:
  • Linux
  • macOS
  • Windows

1. Clone the repository:

git clone https://github.com/fengxsong/httpdispatcher.git
cd httpdispatcher

2. Build the project:

cargo build --release

3. Run the service:

cargo run --release

The configuration file uses YAML format with the default filename config.yaml. The configuration file contains the following main sections:

sources:
  http_input:
    type: http
    address: 0.0.0.0
    port: 3000
    path: /ingest
    max_body_size_bytes: 10485760  # 10MB
    basic_auth: "dXNlcjpwYXNzd29yZA==" # not yet supported

transforms:
  alert_processor:
    type: remap
    inputs: ["http_input"]
    source: |
      .metadata = {
        "processed_at": now(),
        "source_type": "alertmanager",
        "alert_name": .alerts[0].labels.alertname,
        "severity": .alerts[0].labels.severity || "unknown"
      }
      .

sinks:
  webhook_output:
    type: http
    inputs: ["alert_processor"]
    uri: "https://api.example.com/webhooks/alerts"
    method: POST
    template: |
      {
        "id": "{{ id }}",
        "alert": {{ to_json(alerts[0]) }},
        "timestamp": "{{ metadata.processed_at }}",
        "source": "{{ metadata.source_type }}",
        "severity": "{{ metadata.severity }}"
      }
    headers:
      Authorization: "Bearer {{ env.API_TOKEN }}"
      Content-Type: "application/json"
      X-Source: "httpdispatcher"
    timeout_ms: 5000
    retry_attempts: 3
    follow_redirects: true
    tls:
      verify: true
      ca_file: "/path/to/ca.pem"  # Optional

curl -X POST http://localhost:9090/ingest \
  -H "Content-Type: application/json" \
  -d '{"value": "test_event", "data": {"key": "value"}}'

curl -X POST http://localhost:9090/echo \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, World!"}'

Main dependencies include:

• tokio: Async runtime
• axum: Web framework
• vrl: Data transformation language
• serde: Serialization/deserialization
• reqwest: HTTP client
• tracing: Logging
• tower: Middleware framework
• tower-http: HTTP-specific middleware
• tera: Template engine

# Run tests
cargo test

# Run with debug logging
RUST_LOG=debug cargo run

# Build for production
cargo build --release

httpdispatcher/
├── src/
│   ├── main.rs           # Application entry point
│   ├── config/           # Configuration handling
│   ├── server/           # HTTP server implementation
│   ├── transform/        # Data transformation logic
│   └── sink/            # Output handling
├── config.yaml          # Default configuration
├── Cargo.toml          # Project dependencies
└── README.md           # This file

This project is licensed under the MIT License - see the LICENSE file for details.

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Generated with Cursor

• Vector for the VRL language inspiration
• Axum for the excellent web framework
• Tokio for the async runtime