A flexible HTTP client with automatic retry logic using exponential backoff, built with the Functional Options Pattern.

• go-httpretry
  • Table of Contents
  • Features
  • Installation
  • Quick Start
    • Basic Usage (Default Settings)
    • Using Convenience Methods
    • JSON Requests Made Easy
    • Custom Configuration
    • Using Preset Configurations
  • Documentation
    • Key Topics
      • Exponential Backoff
      • Default Retry Behavior
      • Context Support
      • Middleware
    • Complete Working Examples
    • ⚠️ Important: Large File Uploads
  • Testing
  • Design Principles
  • License
  • Author

• Automatic Retries: Retries failed requests with configurable exponential backoff
• Smart Retry Logic: Default retries on network errors, 5xx server errors, and 429 (Too Many Requests)
• Preset Configurations: Ready-to-use presets for common scenarios (realtime, background, rate-limited, microservice, webhook, critical, fast-fail, etc.)
• Middleware Support: Two-level middleware system for per-attempt and request-level customization (rate limiting, circuit breaking, logging, tracing)
• Structured Error Types: Rich error information with RetryError for programmatic error inspection
• Convenience Methods: Simple HTTP methods (Get, Post, Put, Patch, Delete, Head) with optional request configuration
• Request Options: Flexible request configuration with WithBody(), WithJSON(), WithHeader(), and WithHeaders()
• Jitter Support: Optional random jitter to prevent thundering herd problem
• Retry-After Header: Respects HTTP Retry-After header for rate limiting (RFC 2616)
• Observability: Built-in support for metrics collection, distributed tracing, and structured logging (uses standard library log/slog by default, interface-driven for custom implementations)
• Flexible Configuration: Use functional options to customize retry behavior
• Context Support: Respects context cancellation and timeouts
• Custom Retry Logic: Pluggable retry checker for custom retry conditions
• Resource Safe: Automatically closes response bodies before retries to prevent leaks
• Zero Dependencies: Uses only Go standard library

Install the package using go get:

go get github.com/appleboy/go-httpretry

Then import it in your Go code:

import "github.com/appleboy/go-httpretry"

package main

import (
    "context"
    "log"

    "github.com/appleboy/go-httpretry"
)

func main() {
    // Create a retry client with defaults:
    // - 3 max retries
    // - 1 second initial delay
    // - 10 second max delay
    // - 2.0x exponential multiplier
    // - Jitter enabled (±25% randomization)
    // - Retry-After header respected (HTTP standard compliant)
    // - Structured logging to stderr using log/slog (INFO level)
    client, err := retry.NewClient()
    if err != nil {
        log.Fatal(err)
    }

    // Simple GET request
    // Retry operations will be automatically logged to stderr:
    // 2024/02/14 10:00:00 WARN request failed, will retry method=GET attempt=1 reason=5xx
    // 2024/02/14 10:00:00 INFO retrying request method=GET attempt=2 delay=1s
    resp, err := client.Get(context.Background(), "https://api.example.com/data")
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()
}

// GET request
resp, err := client.Get(ctx, "https://api.example.com/users")

// POST request with JSON body (automatic marshaling)
type User struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}
user := User{Name: "John", Email: "[email protected]"}
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithJSON(user))

// POST request with raw JSON body
jsonData := bytes.NewReader([]byte(`{"name":"John"}`))
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithBody("application/json", jsonData))

// PUT request with custom headers
resp, err := client.Put(ctx, "https://api.example.com/users/123",
    retry.WithJSON(user),
    retry.WithHeader("Authorization", "Bearer token"))

// DELETE request
resp, err := client.Delete(ctx, "https://api.example.com/users/123")

The WithJSON() helper automatically marshals your data to JSON:

type CreateUserRequest struct {
    Name  string `json:"name"`
    Email string `json:"email"`
    Age   int    `json:"age"`
}

user := CreateUserRequest{
    Name:  "John Doe",
    Email: "[email protected]",
    Age:   30,
}

// Automatically marshals to JSON and sets Content-Type header
resp, err := client.Post(ctx, "https://api.example.com/users",
    retry.WithJSON(user))

client, err := retry.NewClient(
    retry.WithMaxRetries(5),                           // Retry up to 5 times
    retry.WithInitialRetryDelay(500*time.Millisecond), // Start with 500ms delay
    retry.WithMaxRetryDelay(30*time.Second),           // Cap delay at 30s
    retry.WithRetryDelayMultiple(3.0),                 // Triple delay each time
)
if err != nil {
    log.Fatal(err)
}

The library provides optimized presets for common scenarios:

// Realtime client - Fast response times for user-facing requests
client, err := retry.NewRealtimeClient()

// Background client - Reliable background task processing
client, err := retry.NewBackgroundClient()

// Rate-limited client - Respects API rate limits
client, err := retry.NewRateLimitedClient()

// Microservice client - Internal service communication
client, err := retry.NewMicroserviceClient()

// Critical client - Mission-critical operations (payments, etc.)
client, err := retry.NewCriticalClient()

// Fast-fail client - Health checks and service discovery
client, err := retry.NewFastFailClient()

All presets can be customized by passing additional options:

// Start with realtime preset but use more retries
client, err := retry.NewRealtimeClient(
    retry.WithMaxRetries(5), // Override preset default
)

For detailed documentation, please refer to:

• Preset Configurations - Pre-configured clients for common scenarios (realtime, background, rate-limited, microservice, webhook, critical, fast-fail, etc.)
• Configuration Options - All available configuration options including retry behavior, HTTP client settings, custom TLS, and request options
• Middleware - Two-level middleware system for per-attempt and request-level customization (rate limiting, circuit breaking, logging, custom behaviors)
• Error Handling - Structured error handling with RetryError and response inspection
• Observability - Metrics collection, distributed tracing, and structured logging (OpenTelemetry, Prometheus, slog integration patterns)
• Examples - Detailed usage examples for various scenarios

Retries use exponential backoff to avoid overwhelming the server:

1. First retry: Wait initialRetryDelay (default: 1s)
2. Second retry: Wait initialRetryDelay * multiplier (default: 2s)
3. Third retry: Wait initialRetryDelay * multiplier² (default: 4s)
4. Subsequent retries: Continue multiplying until maxRetryDelay is reached

The DefaultRetryableChecker retries in the following cases:

• Network errors: Connection refused, timeouts, DNS errors, etc.
• 5xx Server Errors: 500, 502, 503, 504, etc.
• 429 Too Many Requests: Rate limiting errors

It does NOT retry:

• 4xx Client Errors (except 429): 400, 401, 403, 404, etc.
• 2xx Success: 200, 201, 204, etc.
• 3xx Redirects: 301, 302, 307, etc.

The client respects context cancellation and timeouts. There are two ways to pass context:

Option 1: Use request's context (recommended)

// Overall timeout for the entire operation (including retries)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

req, _ := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.example.com/data", nil)
resp, err := client.Do(req)
if err != nil {
    // May be context.DeadlineExceeded
    log.Printf("Request failed: %v", err)
}

Option 2: Use DoWithContext for explicit context

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

req, _ := http.NewRequest(http.MethodGet, "https://api.example.com/data", nil)
resp, err := client.DoWithContext(ctx, req)
if err != nil {
    log.Printf("Request failed: %v", err)
}

The library supports a two-level middleware system for adding custom behavior:

Per-Attempt Middleware - Executes for every HTTP attempt including retries:

client, _ := retry.NewClient(
    retry.WithPerAttemptMiddleware(
        retry.LoggingMiddleware(myLogger),        // Logs each attempt
        retry.HeaderMiddleware(map[string]string{ // Adds headers per attempt
            "X-Client-Version": "1.0",
        }),
    ),
)

Request-Level Middleware - Executes once per client call, wrapping the entire retry operation:

client, _ := retry.NewClient(
    retry.WithRequestMiddleware(
        retry.RateLimitMiddleware(myRateLimiter),         // Rate limits entire operation
        retry.CircuitBreakerMiddleware(myCircuitBreaker), // Protects from cascading failures
    ),
)

For detailed documentation, custom middleware examples, and best practices, see Middleware Documentation and _example/middleware.

For complete, runnable examples, see:

• _example/basic - Basic usage with default settings
• _example/advanced - Advanced configuration with custom retry logic
• _example/middleware - Per-attempt and request-level middleware patterns (logging, rate limiting, circuit breaking)
• _example/observability - Metrics, tracing, and logging integration patterns (Prometheus, OpenTelemetry, slog)
• _example/convenience_methods - Using convenience HTTP methods (GET, POST, PUT, DELETE, HEAD, PATCH)
• _example/request_options - Request options usage (WithBody, WithJSON, WithHeader, WithHeaders)
• _example/large_file_upload - ⚠️ Important: Correct way to upload large files (>10MB) with retry support
• example_test.go - Additional examples and test cases

Each example can be run independently:

cd _example/basic && go run main.go
cd _example/advanced && go run main.go
cd _example/middleware && go run main.go
cd _example/observability && go run main.go
cd _example/convenience_methods && go run main.go
cd _example/request_options && go run main.go
cd _example/large_file_upload && go run main.go

Do NOT use WithBody() or WithJSON() for files larger than 10MB. These functions buffer the entire body in memory to support retries.

For large files, use the Do() method with a custom GetBody function:

// ✅ CORRECT: Upload large files with retry support
file, _ := os.Open("large-file.dat")
req, _ := http.NewRequestWithContext(ctx, "POST", url, file)

// CRITICAL: Set GetBody to reopen the file for each retry
req.GetBody = func() (io.ReadCloser, error) {
    return os.Open("large-file.dat")
}

resp, err := client.Do(req)

Size Guidelines:

• ✅ <1MB: Safe to use WithBody() or WithJSON()
• ⚠️ 1-10MB: Use with caution, monitor memory usage
• ❌ >10MB: Use Do() with GetBody (see large_file_upload example)

For complete patterns and best practices, see the large_file_upload example with detailed explanations.

Run the test suite:

go test -v ./...

With coverage:

go test -v -cover ./...

Or use the Makefile:

make test
make lint

• Functional Options Pattern: Provides clean, flexible API for both client configuration and request options
• Sensible Defaults: Works out of the box for most use cases
• Convenience Methods: Simple HTTP methods (Get, Post, Put, Patch, Delete, Head) with optional configuration through RequestOption functions
• Separation of Concerns: HTTP client configuration (including TLS) is the user's responsibility; retry logic is ours
• Single Responsibility: Focus exclusively on retry behavior, not HTTP client building
• Context-Aware: Respects cancellation and timeouts
• Resource Safe: Prevents response body leaks by closing them before retries
• Request Cloning: Clones requests for each retry to handle consumed request bodies
• Zero Dependencies: Uses only standard library

This project is licensed under the MIT License - see the LICENSE file for details.

Copyright (c) 2026 Bo-Yi Wu

• GitHub: @appleboy
• Website: https://blog.wu-boy.com

Support this project: