go-job is a flexible and extensible job scheduling and execution library for Go. It enables you to register, schedule, and manage jobs with arbitrary function signatures, supporting custom executors, priorities, and advanced scheduling options such as cron expressions and delayed execution.

The library provides robust job observation features, including state and log history tracking, as well as customizable response and error handlers. With support for distributed storage backends, go-job is suitable for both local and distributed environments, making it ideal for building scalable, reliable job processing systems in Go applications.

go-job offers a powerful and flexible foundation for job scheduling, execution, and monitoring in Go. It supports arbitrary function execution, dynamic scheduling, prioritization, distributed processing, and detailed job observation.

Key features include:

• Arbitrary Function Execution – Register and run any function with custom input/output types using Go's any type. This allows you to create jobs from functions with any signature, enabling integration of diverse tasks.
• Flexible Scheduling – Schedule jobs to run immediately, after a delay, at a specific time, or on a recurring cron schedule. Supports both one-time and repeated executions (cron), providing versatility for time-based job triggers.
• Queue Priority & Worker Management – Prioritize jobs in the queue and dynamically adjust worker pools for concurrency control. Higher-priority jobs run before lower-priority ones, and the number of worker goroutines can be scaled to meet throughput needs.
• Job Observation – Monitor the full lifecycle of jobs, including state changes and logs. You can attach custom handlers to job completion or failure events and track each job’s state history and log history for auditing and debugging.
• Pluggable Storage for Distributed Support – Abstracted storage via a Store interface allows integration of various backends (in-memory, file, database, etc.). This makes it possible to coordinate jobs across multiple nodes, enabling distributed job processing with persistent state.

For detailed feature descriptions and usage examples, see the Feature Overview and Usage Guide.

go get github.com/cybergarage/go-job

package main

import (
    "fmt"
    "github.com/cybergarage/go-job/job"
)

func main() {
	// Create a job manager
	mgr, _ := job.NewManager()

	// Register a job with a custom executor
	sumJob, _ := job.NewJob(
		job.WithKind("sum"),
		job.WithExecutor(func(ji job.Instance, a, b int) int {
			ji.Infof("%s (%s): attempts %d", ji.UUID(), ji.Kind(), ji.Attempts())
			return a + b
		}),
		job.WithStateChangeProcessor(func(ji job.Instance, state job.JobState) {
			ji.Infof("State changed to: %v", state)
		}),
		job.WithCompleteProcessor(func(ji job.Instance, res []any) {
			ji.Infof("Result: %v", res)
		}),
		job.WithTerminateProcessor(func(ji job.Instance, err error) error {
			ji.Errorf("Error executing job: %v", err)
			return err
		}),
	)
	mgr.RegisterJob(sumJob)

	// Schedule the registered job
	ji, _ := mgr.ScheduleRegisteredJob("sum",
		job.WithArguments("?", 1, 2),   // "?" is a dummy placeholder for job.Instance
		job.WithScheduleAt(time.Now()), // immediate scheduling is the default, so this option is redundant
	)

	// Start the job manager
	mgr.Start()

	// Wait waits for all jobs to complete or terminate.
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()
	mgr.Wait(ctx)

	// Retrieve all queued and executed job instances
	query := job.NewQuery() // queries all job instances (any state)
	jis, _ := mgr.LookupInstances(query)
	for _, ji := range jis {
		fmt.Printf("Job Instance: %s, UUID: %s, State: %s\n", ji.Kind(), ji.UUID(), ji.State())
	}

	// Retrieve and print the job instance state history
	query = job.NewQuery(
		job.WithQueryInstance(ji), // filter by specific job instance
	)
	history, _ := mgr.LookupInstanceHistory(query)
	for _, record := range history {
		fmt.Println(record.State())
	}

	// Retrieve and print the job instance logs
	query = job.NewQuery(
		job.WithQueryInstance(ji), // filter by specific job instance
	)
	logs, _ := mgr.LookupInstanceLogs(query)
	for _, log := range logs {
		fmt.Println(log.Message())
	}

	// Stop the job manager
	mgr.Stop()
}

• Getting Started
  • Quick Start
• Usage & Features
  • Feature Overview and Usage Guide
• Operation
  • Command-Line Interface (jobctl)
  • gRPC API
  • Metrics (Prometheus)
• Design & Comparison
  • go-job Comparison (OpenAI Research)

• Architecture & Development
  • Design and Architecture
• Extending go-job
  • Extension Guide

go-job is developed in collaboration with the following Cybergarage projects:

• go-logger
• go-safecast