A high-performance, feature-rich Go JSON processing library with 100%
encoding/jsoncompatibility, providing powerful path operations, type safety, performance optimization, and rich advanced features.
π δΈζζζ‘£ - User guide
- π Overview
- π Quick Start
- β‘ Core Features
- π Performance Benchmarks
- π― Path Expressions
- π§ Configuration Options
- π File Operations
- π Data Validation
- π― Use Cases
- π API Reference
- π‘οΈ Error Handling Guide
- π‘ Examples & Resources
cybergodev/json is a high-performance Go JSON processing library that maintains 100% compatibility with the standard encoding/json package while providing powerful path operations, type safety, performance optimization, and rich advanced features.
- π Full Compatibility - 100% compatible with standard
encoding/json, zero learning curve, drop-in replacement - π― Powerful Paths - Support for complex path expressions, complete complex data operations in one line
- π High Performance - Smart caching, concurrent safety, memory optimization, production-ready performance
- π‘οΈ Type Safety - Generic support, compile-time checking, intelligent type conversion
- π§ Feature Rich - Batch operations, data validation, file operations, performance monitoring
- ποΈ Production Ready - Thread-safe, error handling, security configuration, monitoring metrics
- π API Data Processing - Fast extraction and transformation of complex response data
- βοΈ Configuration Management - Dynamic configuration reading and batch updates
- π Data Analysis - Statistics and analysis of large amounts of JSON data
- π Microservice Communication - Data exchange and format conversion between services
- π Log Processing - Parsing and analysis of structured logs
- π Examples - Comprehensive code examples for all features
- βοΈ Configuration Guide - Advanced configuration and optimization
- π Compatibility - Compatibility guide and migration information
- β‘ Quick Reference - Quick reference guide for common features
Our library maintains 100% compatibility while offering significant performance improvements:
| Operation | cybergodev/json | encoding/json | Improvement |
|---|---|---|---|
| Marshal (small) | 1.2 Β΅s/op | 1.5 Β΅s/op | 25% faster β‘ |
| Marshal (large) | 45 Β΅s/op | 58 Β΅s/op | 22% faster β‘ |
| Unmarshal (small) | 2.1 Β΅s/op | 2.8 Β΅s/op | 33% faster β‘ |
| Unmarshal (large) | 78 Β΅s/op | 95 Β΅s/op | 18% faster β‘ |
| Path Get (cached) | 0.3 Β΅s/op | N/A | New feature π― |
| Path Get (uncached) | 0.8 Β΅s/op | N/A | New feature π― |
// Memory allocation comparison
BenchmarkMarshal/cybergodev-json 1000000 1234 ns/op 512 B/op 8 allocs/op
BenchmarkMarshal/encoding-json 800000 1567 ns/op 768 B/op 12 allocs/op
// Results:
// - 33% fewer allocations
// - 40% less memory per operation
// - 85-95% cache hit ratio in typical workloads- Thread-safe operations: Zero performance penalty
- Concurrent throughput: 10,000+ operations/second
- Cache efficiency: 85-95% hit ratio in production workloads
- Memory safety: Zero memory leaks in stress tests
# Run all benchmarks
go test -bench=. -benchmem
# Run specific benchmarks
go test -bench=BenchmarkMarshal -benchmem
go test -bench=BenchmarkGet -benchmem
go test -bench=BenchmarkConcurrent -benchmemNote: Benchmarks run on: Intel i7-9700K, 16GB RAM, Go 1.24, Windows 11
| Syntax | Description | Example | Result |
|---|---|---|---|
. |
Property access | user.name |
Get user's name property |
[n] |
Array index | users[0] |
Get first user |
[-n] |
Negative index | users[-1] |
Get last user |
[start:end:step] |
Array slice | users[1:3] |
Get users at index 1-2 |
{field} |
Batch extract | users{name} |
Extract all user names |
{flat:field} |
Flatten extract | users{flat:skills} |
Flatten extract all skills |
go get github.com/cybergodev/jsonpackage main
import (
"fmt"
"github.com/cybergodev/json"
)
func main() {
// 1. Full compatibility with standard library
data := map[string]any{"name": "Alice", "age": 25}
jsonBytes, _ := json.Marshal(data)
var result map[string]any
json.Unmarshal(jsonBytes, &result)
// 2. Powerful path operations (enhanced features)
jsonStr := `{"user":{"profile":{"name":"Alice","age":25}}}`
name, _ := json.GetString(jsonStr, "user.profile.name")
fmt.Println(name) // "Alice"
age, _ := json.GetInt(jsonStr, "user.profile.age")
fmt.Println(age) // 25
}// Complex JSON data
complexData := `{
"users": [
{"name": "Alice", "skills": ["Go", "Python"], "active": true},
{"name": "Bob", "skills": ["Java", "React"], "active": false}
]
}`
// Get all user names
names, _ := json.Get(complexData, "users{name}")
// Result: ["Alice", "Bob"]
// Get all skills (flattened)
skills, _ := json.Get(complexData, "users{flat:skills}")
// Result: ["Go", "Python", "Java", "React"]
// Batch get multiple values
paths := []string{"users[0].name", "users[1].name", "users{active}"}
results, _ := json.GetMultiple(complexData, paths)// Basic retrieval
json.Get(data, "user.name") // Get any type
json.GetString(data, "user.name") // Get string
json.GetInt(data, "user.age") // Get integer
json.GetBool(data, "user.active") // Get boolean
json.GetArray(data, "user.tags") // Get array
json.GetObject(data, "user.profile") // Get object
// Type-safe retrieval
json.GetTyped[string](data, "user.name") // Generic type safety
json.GetTyped[[]User](data, "users") // Custom types
// Retrieval with default values
json.GetStringWithDefault(data, "user.name", "Anonymous")
json.GetIntWithDefault(data, "user.age", 0)
// Batch retrieval
paths := []string{"user.name", "user.age", "user.email"}
results, _ := json.GetMultiple(data, paths)// Basic setting - returns modified data on success, original data on failure
result, err := json.Set(data, "user.name", "Alice")
if err != nil {
// result contains original unmodified data
fmt.Printf("Set failed: %v, original data preserved\n", err)
} else {
// result contains modified data
fmt.Println("Set successful, data modified")
}
// Auto-create paths
result, err := json.SetWithAdd(data, "user.profile.city", "NYC")
if err != nil {
// result contains original data if creation failed
fmt.Printf("Path creation failed: %v\n", err)
}
// Batch setting
updates := map[string]any{
"user.name": "Bob",
"user.age": 30,
"user.active": true,
}
result, err := json.SetMultiple(data, updates)
// Same behavior: success = modified data, failure = original datajson.Delete(data, "user.temp") // Delete field
json.DeleteWithCleanNull(data, "user.temp") // Delete and cleanup nulls// Basic iteration - read-only traversal
json.Foreach(data, func (key any, item *json.IterableValue) {
name := item.GetString("name")
fmt.Printf("Key: %v, Name: %s\n", key, name)
})
// Path iteration - read-only traversal of JSON subset
json.ForeachWithPath(data, "data.list.users", func (key any, user *json.IterableValue) {
name := user.GetString("name")
age := user.GetInt("age")
// Note: ForeachWithPath is read-only, modifications won't affect original data
fmt.Printf("User: %s, Age: %d\n", name, age)
})
// Iterate and return modified JSON - supports data modification
modifiedJson, err := json.ForeachReturn(data, func (key any, item *json.IterableValue) {
// Modify data during iteration
if item.GetString("status") == "inactive" {
item.Set("status", "active")
item.Set("updated_at", time.Now().Format("2006-01-02"))
}
// Batch update user information
if key == "users" {
item.SetMultiple(map[string]any{
"last_login": time.Now().Unix(),
"version": "2.0",
})
}
})complexData := `{
"company": {
"departments": [
{
"name": "Engineering",
"teams": [
{
"name": "Backend",
"members": [
{"name": "Alice", "skills": ["Go", "Python"], "level": "Senior"},
{"name": "Bob", "skills": ["Java", "Spring"], "level": "Mid"}
]
}
]
}
]
}
}`
// Multi-level nested extraction
allMembers, _ := json.Get(complexData, "company.departments{teams}{flat:members}")
// Result: [Alice's data, Bob's data]
// Extract specific fields
allNames, _ := json.Get(complexData, "company.departments{teams}{flat:members}{name}")
// Result: ["Alice", "Bob"]
// Flatten skills extraction
allSkills, _ := json.Get(complexData, "company.departments{teams}{flat:members}{flat:skills}")
// Result: ["Go", "Python", "Java", "Spring"]arrayData := `{
"numbers": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
"users": [
{"name": "Alice", "age": 25},
{"name": "Bob", "age": 30}
]
}`
// Array indexing and slicing
first, _ := json.GetInt(arrayData, "numbers[0]") // 1
last, _ := json.GetInt(arrayData, "numbers[-1]") // 10 (negative index)
slice, _ := json.Get(arrayData, "numbers[1:4]") // [2, 3, 4]
everyOther, _ := json.Get(arrayData, "numbers[::2]") // [1, 3, 5, 7, 9]
everyOther, _ := json.Get(arrayData, "numbers[::-2]") // [10 8 6 4 2]
// Nested array access
ages, _ := json.Get(arrayData, "users{age}") // [25, 30]The json.New() function now supports optional configuration parameters:
// 1. No parameters - uses default configuration
processor1 := json.New()
defer processor1.Close()
// 2. Explicit nil - same as default configuration
processor2 := json.New()
defer processor2.Close()
// 3. Custom configuration
customConfig := &json.Config{
// Cache settings
EnableCache: true, // Enable cache
MaxCacheSize: 5000, // Cache entry count
CacheTTL: 10 * time.Minute, // Cache expiration time
// Size limits
MaxJSONSize: 50 * 1024 * 1024, // 50MB JSON size limit
MaxPathDepth: 200, // Path depth limit
MaxBatchSize: 2000, // Batch operation size limit
// Concurrency settings
MaxConcurrency: 100, // Maximum concurrency
ParallelThreshold: 20, // Parallel processing threshold
// Processing options
EnableValidation: true, // Enable validation
StrictMode: false, // Non-strict mode
CreatePaths: true, // Auto-create paths
CleanupNulls: true, // Cleanup null values
}
processor3 := json.New(customConfig)
defer processor3.Close()
// 4. Predefined configurations
secureProcessor := json.New(json.HighSecurityConfig())
largeDataProcessor := json.New(json.LargeDataConfig())opts := &json.ProcessorOptions{
CreatePaths: true, // Auto-create paths
CleanupNulls: true, // Cleanup null values
CompactArrays: true, // Compact arrays
ContinueOnError: false, // Continue on error
MaxDepth: 50, // Maximum depth
}
result, _ := json.Get(data, "path", opts)processor := json.New(json.DefaultConfig())
defer processor.Close()
// Get statistics after operations
stats := processor.GetStats()
fmt.Printf("Total operations: %d\n", stats.OperationCount)
fmt.Printf("Cache hit rate: %.2f%%\n", stats.HitRatio*100)
fmt.Printf("Cache memory usage: %d bytes\n", stats.CacheMemory)
// Get health status
health := processor.GetHealthStatus()
fmt.Printf("System health: %v\n", health.Healthy)// Load JSON from file
data, err := json.LoadFromFile("config.json")
if err != nil {
log.Printf("File load failed: %v", err)
return
}
// Save to file (pretty format)
err = json.SaveToFile("output_pretty.json", data, true)
// Save to file (compact format)
err = json.SaveToFile("output.json", data, false)
// Load from Reader
file, err := os.Open("large_data.json")
if err != nil {
log.Fatal(err)
}
defer file.Close()
data, err = json.LoadFromReader(file)
// Save to Writer
var buffer bytes.Buffer
err = json.SaveToWriter(&buffer, data, true)configFiles := []string{
"database.json",
"cache.json",
"logging.json",
}
allConfigs := make(map[string]any)
for _, filename := range configFiles {
config, err := json.LoadFromFile(filename)
if err != nil {
log.Printf("Loading %s failed: %v", filename, err)
continue
}
configName := strings.TrimSuffix(filename, ".json")
allConfigs[configName] = config
}
// Save merged configuration
err := json.SaveToFile("merged_config.json", allConfigs, true)// Define JSON Schema
schema := &json.Schema{
Type: "object",
Properties: map[string]*json.Schema{
"name": (&json.Schema{
Type: "string",
}).SetMinLength(1).SetMaxLength(100),
"age": (&json.Schema{
Type: "number",
}).SetMinimum(0.0).SetMaximum(150.0),
"email": {
Type: "string",
Format: "email",
},
},
Required: []string{"name", "age", "email"},
}
// Validate data
testData := `{
"name": "Alice",
"age": 25,
"email": "[email protected]"
}`
processor := json.New(json.DefaultConfig())
errors, err := processor.ValidateSchema(testData, schema)
if len(errors) > 0 {
fmt.Println("Validation errors:")
for _, validationErr := range errors {
fmt.Printf(" Path %s: %s\n", validationErr.Path, validationErr.Message)
}
} else {
fmt.Println("Data validation passed")
}// Security configuration
secureConfig := &json.Config{
MaxJSONSize: 10 * 1024 * 1024, // 10MB JSON size limit
MaxPathDepth: 50, // Path depth limit
MaxNestingDepth: 100, // Object nesting depth limit
MaxArrayElements: 10000, // Array element count limit
MaxObjectKeys: 1000, // Object key count limit
ValidateInput: true, // Input validation
EnableValidation: true, // Enable validation
StrictMode: true, // Strict mode
}
processor := json.New(secureConfig)
defer processor.Close()// Typical REST API response
apiResponse := `{
"status": "success",
"code": 200,
"data": {
"users": [
{
"id": 1,
"profile": {
"name": "Alice Johnson",
"email": "[email protected]"
},
"permissions": ["read", "write", "admin"],
"metadata": {
"created_at": "2023-01-15T10:30:00Z",
"tags": ["premium", "verified"]
}
}
],
"pagination": {
"page": 1,
"total": 25
}
}
}`
// Quick extraction of key information
status, _ := json.GetString(apiResponse, "status")
code, _ := json.GetInt(apiResponse, "code")
// Batch extract user information
userNames, _ := json.Get(apiResponse, "data.users.profile.name")
// Result: ["Alice Johnson"]
userEmails, _ := json.Get(apiResponse, "data.users.profile.email")
// Result: ["[email protected]"]
// Flatten extract all permissions
allPermissions, _ := json.Get(apiResponse, "data.users{flat:permissions}")
// Result: ["read", "write", "admin"]
// Get pagination information
totalUsers, _ := json.GetInt(apiResponse, "data.pagination.total")
currentPage, _ := json.GetInt(apiResponse, "data.pagination.page")
fmt.Printf("Status: %s (Code: %d)\n", status, code)
fmt.Printf("Total users: %d, Current page: %d\n", totalUsers, currentPage)// Multi-environment configuration file
configJSON := `{
"app": {
"name": "MyApplication",
"version": "1.2.3"
},
"environments": {
"development": {
"database": {
"host": "localhost",
"port": 5432,
"name": "myapp_dev"
},
"cache": {
"enabled": true,
"host": "localhost",
"port": 6379
}
},
"production": {
"database": {
"host": "prod-db.example.com",
"port": 5432,
"name": "myapp_prod"
},
"cache": {
"enabled": true,
"host": "prod-cache.example.com",
"port": 6379
}
}
}
}`
// Type-safe configuration retrieval
dbHost := json.GetStringWithDefault(configJSON, "environments.production.database.host", "localhost")
dbPort := json.GetIntWithDefault(configJSON, "environments.production.database.port", 5432)
cacheEnabled := json.GetBoolWithDefault(configJSON, "environments.production.cache.enabled", false)
fmt.Printf("Production database: %s:%d\n", dbHost, dbPort)
fmt.Printf("Cache enabled: %v\n", cacheEnabled)
// Dynamic configuration updates
updates := map[string]any{
"app.version": "1.2.4",
"environments.production.cache.ttl": 10800, // 3 hours
}
newConfig, _ := json.SetMultiple(configJSON, updates)// Log and monitoring data
analyticsData := `{
"events": [
{
"type": "request",
"user_id": "user_123",
"endpoint": "/api/users",
"status_code": 200,
"response_time": 45
},
{
"type": "error",
"user_id": "user_456",
"endpoint": "/api/orders",
"status_code": 500,
"response_time": 5000
}
]
}`
// Extract all event types
eventTypes, _ := json.Get(analyticsData, "events.type")
// Result: ["request", "error"]
// Extract all status codes
statusCodes, _ := json.Get(analyticsData, "events.status_code")
// Result: [200, 500]
// Extract all response times
responseTimes, _ := json.GetTyped[[]float64](analyticsData, "events.response_time")
// Result: [45, 5000]
// Calculate average response time
times := responseTimes
var total float64
for _, t := range times {
total += t
}
avgTime := total / float64(len(times))
fmt.Printf("Average response time: %.2f ms\n", avgTime)// Basic retrieval
json.Get(data, path) (any, error)
json.GetString(data, path) (string, error)
json.GetInt(data, path) (int, error)
json.GetBool(data, path) (bool, error)
json.GetFloat64(data, path) (float64, error)
json.GetArray(data, path) ([]any, error)
json.GetObject(data, path) (map[string]any, error)
// Type-safe retrieval
json.GetTyped[T](data, path) (T, error)
// Retrieval with default values
json.GetStringWithDefault(data, path, defaultValue) string
json.GetIntWithDefault(data, path, defaultValue) int
json.GetBoolWithDefault(data, path, defaultValue) bool
// Batch retrieval
json.GetMultiple(data, paths) (map[string]any, error)// Basic setting - improved error handling
// Returns: (modified_data, nil) on success, (original_data, error) on failure
json.Set(data, path, value) (string, error)
json.SetWithAdd(data, path, value) (string, error)
// Batch setting - same improved behavior
json.SetMultiple(data, updates) (string, error)
json.SetMultipleWithAdd(data, updates) (string, error)json.Delete(data, path) (string, error)
json.DeleteWithCleanNull(data, path) (string, error)// Basic iteration methods
json.Foreach(data, callback) error
json.ForeachReturn(data, callback) (string, error)
// Path iteration - read-only traversal of specified path data
json.ForeachWithPath(data, path, callback) error
// Nested iteration - prevents state conflicts
json.ForeachNested(data, callback) error
json.ForeachReturnNested(data, callback) (string, error)
// IterableValue nested methods - used within iteration callbacks
item.ForeachReturnNested(path, callback) errorUse case comparison:
| Method | Return Value | Data Modification | Traversal Range | Usage Scenarios |
|---|---|---|---|---|
Foreach |
error |
Not allow | Complete JSON | Read-only traversal of the entire JSON |
ForeachWithPath |
error |
Not allow | Specified path | Read-only traversal of JSON subset |
ForeachReturn |
(string, error) |
Allow | Complete JSON | Data modification, batch update |
// File read/write
json.LoadFromFile(filename, ...opts) (string, error)
json.SaveToFile(filename, data, pretty) error
// Stream operations
json.LoadFromReader(reader, ...opts) (string, error)
json.SaveToWriter(writer, data, pretty) error// Schema validation
processor.ValidateSchema(data, schema) ([]ValidationError, error)
// Basic validation
json.Valid(data) bool// Create processor
json.New(config) *Processor
json.DefaultConfig() *Config
// Cache operations
processor.WarmupCache(data, paths) (*WarmupResult, error)
processor.ClearCache()
// Statistics
processor.GetStats() *Stats
processor.GetHealthStatus() *HealthStatus// Recommended error handling approach
result, err := json.GetString(data, "user.name")
if err != nil {
log.Printf("Failed to get username: %v", err)
// Use default value or return error
return "", err
}
// Use methods with default values
name := json.GetStringWithDefault(data, "user.name", "Anonymous")All Set operations follow a safe-by-default pattern that ensures your data is never corrupted:
// β
Success: Returns modified data
result, err := json.Set(data, "user.name", "Alice")
if err == nil {
// result contains successfully modified JSON
fmt.Println("Data updated:", result)
}
// β Failure: Returns original unmodified data
result, err := json.Set(data, "invalid[path", "value")
if err != nil {
// result still contains valid original data
// Your original data is NEVER corrupted
fmt.Printf("Set failed: %v\n", err)
fmt.Println("Original data preserved:", result)
}Key Benefits:
- π Data Integrity: Original data never corrupted on error
- β Safe Fallback: Always have valid JSON to work with
- π― Predictable: Consistent behavior across all operations
// 1. Path Not Found - Use default values
name := json.GetStringWithDefault(data, "user.name", "Anonymous")
age := json.GetIntWithDefault(data, "user.age", 0)
// 2. Type Mismatch - Check error type
value, err := json.GetInt(data, "user.name") // name is string
if err != nil {
if errors.Is(err, json.ErrTypeMismatch) {
log.Printf("Type mismatch: %v", err)
}
}
// 3. Invalid JSON - Validate first
if !json.Valid([]byte(jsonStr)) {
return fmt.Errorf("invalid JSON input")
}
// 4. Size Limits - Configure appropriately
config := json.DefaultConfig()
config.MaxJSONSize = 50 * 1024 * 1024 // 50MB
processor := json.New(config)
defer processor.Close()jsonData := `{"user": {"name": "Alice", "age": null}}`
// GetInt on null returns 0 and error
age, err := json.GetInt(jsonData, "user.age")
// age = 0, err != nil
// Use Get to check for null explicitly
value, _ := json.Get(jsonData, "user.age")
if value == nil {
fmt.Println("Age is null")
}jsonData := `{"user": {"name": "Alice"}}`
// Missing field
email, err := json.GetString(jsonData, "user.email")
// err = ErrPathNotFound
// Null field
jsonData2 := `{"user": {"name": "Alice", "email": null}}`
email2, err2 := json.GetString(jsonData2, "user.email")
// email2 = "", err2 = nil (null converts to empty string)jsonData := `{"users": [{"name": "Alice"}, {"name": "Bob"}]}`
// Valid index
user, _ := json.Get(jsonData, "users[0]") // OK
// Invalid index
user, err := json.Get(jsonData, "users[10]")
// err = ErrPathNotFound
// Use negative index for last element
lastUser, _ := json.Get(jsonData, "users[-1]") // Gets BobThe repository includes comprehensive examples demonstrating various features and use cases:
- Basic Usage - Fundamental operations and getting started
- JSON Get Operations - Data retrieval examples with different path expressions
- JSON Set Operations - Data modification and batch updates
- JSON Delete Operations - Data deletion and cleanup operations
- File Operations - File I/O, batch processing, and stream operations
- JSON Iteration - Data iteration and traversal patterns
- Flat Extraction - Complex data extraction and flattening
- JSON Encoding - Custom encoding configurations and formatting
- Configuration Management - Processor configuration and optimization
- Compatibility Examples - Drop-in replacement demonstrations
This project is licensed under the MIT License - see the LICENSE file for details.
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
If you find this project useful, please consider giving it a star! β
Made with β€οΈ by the CyberGoDev team