Thanks to visit codestin.com
Credit goes to Github.com

Skip to content

Bareuptime/shipper

Repository files navigation

Shipper Deployment Service

Test Suite Build Matrix Docker Build

A lightweight Go service that acts as a secure deployment gateway for HashiCorp Nomad clusters. This service provides a simple REST API for triggering and monitoring service deployments through Nomad.

Features

  • Secure Authentication: API key-based authentication for deployment requests
  • Deployment Tracking: SQLite database for tracking deployment status and history
  • Health Monitoring: Built-in health check endpoint
  • Service Validation: Optional whitelist of allowed services for enhanced security
  • Monitoring Integration: Optional New Relic integration
  • Docker Ready: Full containerization support with Docker Compose
  • Hot Reload Development: Development environment with hot reload capabilities

πŸ“‹ API Endpoints

Health Check

GET /health

Returns the service health status.

Deploy Service

POST /deploy
Content-Type: application/json
X-Secret-Key: your-64-character-secret-key

{
  "service_name": "my-service",
  "tag_id": "sha-id"
}

Triggers a deployment for the specified service.

Deploy with Job File

POST /deploy/job
Content-Type: multipart/form-data
X-Secret-Key: your-64-character-secret-key

Form data:
- tag_id: sha-id-123
- job_file: (Nomad job file upload, max 1MB)

Uploads and deploys a custom Nomad job file.

Check Deployment Status

GET /status/{tag_id}
X-Secret-Key: your-64-character-secret-key

Returns the status of a specific deployment by its tag ID.

πŸ“š Documentation

Comprehensive documentation and examples are available in the docs/ directory:

βš™οΈ Configuration

The service is configured through environment variables. Copy .env.example to .env and modify as needed:

cp .env.example .env

Environment Variables

Variable Description Default Required
NOMAD_URL Nomad API URL https://your-nomad-cluster:4646 βœ…
NOMAD_TOKEN Nomad API token - βœ…
RPC_SECRET Secret key for API authentication (64 chars) - βœ…
PORT Server port 16166 ❌
SKIP_TLS_VERIFY Skip TLS verification for Nomad API false ❌
LOG_LEVEL Logging level (debug, info, warn, error) info ❌
LOG_FORMAT Log format (json, text) json ❌
NEW_RELIC_ENABLED Enable New Relic monitoring false ❌
NEW_RELIC_LICENSE_KEY New Relic license key - ❌
NEW_RELIC_APP_NAME New Relic application name shipper-deployment ❌

πŸš€ Quick Start

Prerequisites

  • Go 1.24+ (for local development)
  • Docker and Docker Compose (for containerized deployment)
  • Access to a Nomad cluster

Local Development

  1. Clone the repository

    git clone <repository-url>
    cd bastion-deployment
  2. Set up environment variables

    cp .env.example .env
    # Edit .env with your configuration
  3. Run locally

    make run

Docker Development

For development with hot reload (using Air inside Docker):

make dev

This starts the service with Docker Compose and automatically reloads on code changes using the .air.toml configuration.

Production Deployment

  1. Build the Docker image

    make docker-build
  2. Run with Docker Compose

    make docker-run

🎯 Nomad Deployment

For deploying the Shipper service itself in a Nomad cluster:

  1. Basic Nomad Deployment

    # Use the provided Nomad job file
    nomad job run docs/nomad-jobs/shipper-basic.nomad
  2. Production Nomad Deployment

    # For production with high availability
    nomad job run docs/nomad-jobs/shipper-production.nomad
  3. Configure Nomad Variables

    # Set up secrets using Nomad variables
    nomad var put nomad/jobs/shipper-deployment \
      nomad_token="your-nomad-token" \
      rpc_secret="your-64-character-secret"

πŸ“š For detailed Nomad deployment instructions, see docs/nomad-deployment.md

πŸ› οΈ Development

Running Tests

The project includes comprehensive tests covering models, configuration, database operations, handlers, and integration scenarios:

make test

Tests are located in the test/ directory and include:

  • Unit tests: Config, models, database operations
  • Handler tests: API endpoint functionality with mocked dependencies
  • Integration tests: Full HTTP server testing with authentication

CI/CD Pipeline

The project includes comprehensive GitHub Actions workflows:

  • Test Suite (test.yml): Runs tests, checks coverage, formatting, and security
  • Build Matrix (build-matrix.yml): Tests compatibility across Go versions and OS platforms
  • Docker Build (docker.yml): Builds and publishes Docker images

All workflows run on:

  • Every push to main/master branches
  • Every pull request
  • Support for Go 1.22, 1.23, and 1.24
  • Cross-platform testing (Linux, macOS, Windows)

Available Make Commands

make build      # Build the application binary
make run        # Run the application locally
make dev        # Start development environment with hot reload
make test       # Run tests
make fmt        # Format code
make tidy       # Tidy Go modules
make clean      # Clean build artifacts
make docker-build    # Build Docker image
make docker-run      # Run with Docker Compose
make docker-stop     # Stop Docker Compose

Project Structure

.
β”œβ”€β”€ .github/
β”‚   └── workflows/      # CI/CD workflows
β”œβ”€β”€ cmd/
β”‚   └── shipper/        # Application entry point
β”œβ”€β”€ docs/               # Comprehensive documentation
β”‚   β”œβ”€β”€ examples/       # Sample Nomad job files
β”‚   β”œβ”€β”€ nomad-jobs/     # Shipper deployment templates
β”‚   β”œβ”€β”€ api-usage.md    # API documentation and examples
β”‚   β”œβ”€β”€ integration-patterns.md # CI/CD integration patterns
β”‚   β”œβ”€β”€ nomad-deployment.md # Nomad deployment guide
β”‚   └── README.md       # Documentation index
β”œβ”€β”€ internal/
β”‚   β”œβ”€β”€ config/         # Configuration management
β”‚   β”œβ”€β”€ database/       # Database operations
β”‚   β”œβ”€β”€ handlers/       # HTTP handlers
β”‚   β”œβ”€β”€ logger/         # Logging setup
β”‚   β”œβ”€β”€ models/         # Data models
β”‚   β”œβ”€β”€ newrelic/       # New Relic integration
β”‚   β”œβ”€β”€ nomad/          # Nomad client
β”‚   └── server/         # HTTP server setup
β”œβ”€β”€ test/               # Comprehensive test suite
β”œβ”€β”€ .env.example        # Environment variables template
β”œβ”€β”€ .golangci.yml       # Linting configuration
β”œβ”€β”€ docker-compose.yml  # Production Docker Compose
β”œβ”€β”€ docker-compose.dev.yml # Development Docker Compose
└── Makefile           # Build automation

πŸ”’ Security Considerations

  • Change Default Secrets: Always use a secure 64-character secret key in production
  • TLS Verification: Keep SKIP_TLS_VERIFY=false in production environments
  • Network Security: Ensure proper network policies for Nomad cluster access
  • Environment Variables: Never commit .env files with real credentials

πŸ“Š Monitoring

The service includes optional New Relic integration for application performance monitoring. Enable by setting:

NEW_RELIC_ENABLED=true
NEW_RELIC_LICENSE_KEY=your-license-key
NEW_RELIC_APP_NAME=shipper-deployment

🀝 Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add some amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

πŸ“ License

This project is licensed under the MIT License - see the LICENSE file for details.

πŸ› Troubleshooting

Common Issues

  1. Connection to Nomad fails

    • Verify NOMAD_URL and NOMAD_TOKEN are correct
    • Check network connectivity to Nomad cluster
    • Verify TLS settings with SKIP_TLS_VERIFY
  2. Authentication errors

    • Ensure RPC_SECRET is exactly 64 characters
    • Verify the secret key matches in deployment requests
  3. Service deployment fails

    • Verify Nomad token has sufficient permissions
    • Check Nomad cluster status and resources

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

Packages

No packages published