Original TOON (TypeScript/JavaScript) - The original implementation
A Go implementation of Token-Oriented Object Notation (TOON) - a compact, human-readable format designed for passing structured data to Large Language Models with significantly reduced token usage.
TOON saves ~50-60% of tokens compared to standard JSON while remaining readable and parseable by LLMs. It's perfect for:
- 💰 Reducing LLM API costs
- 📊 Passing large datasets to LLMs
- 🚀 Optimizing context window usage
- 🤖 Structured data in AI applications
go get github.com/rumpl/toonpackage main
import (
"fmt"
"github.com/rumpl/toon"
)
func main() {
data := map[string]any{
"users": []any{
map[string]any{"id": 1, "name": "Alice", "role": "admin"},
map[string]any{"id": 2, "name": "Bob", "role": "user"},
},
"total": 2,
}
result, _ := toon.Encode(data)
fmt.Println(result)
}Output:
total: 2
users[2]{id,name,role}:
1,Alice,admin
2,Bob,user
Compare this to the equivalent JSON (123 tokens vs 53 TOON tokens = 57% reduction):
{
"users": [
{ "id": 1, "name": "Alice", "role": "admin" },
{ "id": 2, "name": "Bob", "role": "user" }
],
"total": 2
}- Token-efficient: 50-65% fewer tokens than JSON
- LLM-friendly: Explicit lengths and field lists help models validate output
- Minimal syntax: Removes redundant punctuation
- Indentation-based: Clean, readable structure
- Tabular arrays: Efficient format for uniform data
- Struct support: Works with Go structs using
jsontags - Customizable: Support for different delimiters (comma, tab, pipe)
data := map[string]any{
"id": 123,
"name": "Ada",
"active": true,
}
toon.Encode(data)
// id: 123
// name: Ada
// active: truedata := map[string]any{
"user": map[string]any{
"id": 123,
"name": "Ada",
},
}
toon.Encode(data)
// user:
// id: 123
// name: Adadata := map[string]any{
"tags": []string{"reading", "gaming", "coding"},
}
toon.Encode(data)
// tags[3]: reading,gaming,codingWhen all objects share the same primitive fields, TOON uses an efficient tabular format:
items := []any{
map[string]any{"sku": "A1", "qty": 2, "price": 9.99},
map[string]any{"sku": "B2", "qty": 1, "price": 14.5},
}
data := map[string]any{"items": items}
toon.Encode(data)
// items[2]{price,qty,sku}:
// 9.99,2,A1
// 14.5,1,B2TOON works seamlessly with Go structs:
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Active bool `json:"active"`
}
user := User{ID: 123, Name: "Ada", Active: true}
result, _ := toon.Encode(user)
// active: true
// id: 123
// name: Adadata := []any{"x", "y", "z"}
toon.Encode(data)
// [3]: x,y,zUse tab or pipe delimiters for additional token savings:
opts := toon.EncodeOptions{
Delimiter: toon.TabDelimiter,
}
result, _ := toon.EncodeWithOptions(data, opts)
// items[2 ]{name price}:
// Widget 9.99
// Gadget 14.50opts := toon.EncodeOptions{
Delimiter: toon.PipeDelimiter,
}
result, _ := toon.EncodeWithOptions(data, opts)
// items[2|]{name|price}:
// Widget|9.99
// Gadget|14.50Add a # prefix to array lengths for clarity:
opts := toon.EncodeOptions{
LengthMarker: true,
}
result, _ := toon.EncodeWithOptions(data, opts)
// tags[#3]: reading,gaming,codingopts := toon.EncodeOptions{
Indent: 4,
}
result, _ := toon.EncodeWithOptions(data, opts)
// user:
// id: 123
// name: Adatype EncodeOptions struct {
// Indent is the number of spaces per indentation level (default: 2)
Indent int
// Delimiter is the delimiter for array values and tabular rows
// Options: CommaDelimiter (default), TabDelimiter, PipeDelimiter
Delimiter Delimiter
// LengthMarker adds a # prefix to array lengths (default: false)
LengthMarker bool
}TOON automatically handles various Go types:
| Go Type | TOON Output |
|---|---|
int, float64 |
Numbers without quotes |
string |
Quoted if needed (special chars, looks like number/bool) |
bool |
true or false |
nil |
null |
time.Time |
ISO 8601 string |
*big.Int |
Decimal digits (unquoted) |
NaN, ±Infinity |
null |
-0 |
0 |
TOON quotes strings only when necessary:
Strings that need quoting:
- Empty strings:
"" - Contains delimiter, colon, or quotes:
"a,b","a:b" - Leading/trailing spaces:
" padded " - Looks like boolean/number:
"true","42" - Contains control characters:
"line1\nline2"
Strings safe unquoted:
- Simple text:
hello - Unicode & emoji:
café,🚀 - Inner spaces:
hello world
Encodes a value using default options (2-space indent, comma delimiter, no length marker).
result, err := toon.Encode(data)Encodes a value with custom options.
result, err := toon.EncodeWithOptions(data, toon.EncodeOptions{
Indent: 4,
Delimiter: toon.TabDelimiter,
LengthMarker: true,
})analytics := []any{
map[string]any{"date": "2025-01-01", "views": 1234, "clicks": 89},
map[string]any{"date": "2025-01-02", "views": 2345, "clicks": 156},
map[string]any{"date": "2025-01-03", "views": 1890, "clicks": 123},
}
toonData, _ := toon.Encode(map[string]any{"metrics": analytics})
prompt := fmt.Sprintf(`Analyze this website data:
%s
What trends do you see?`, toonData)
// Send to LLM - uses ~55% fewer tokens than JSON!logEntry := map[string]any{
"timestamp": time.Now(),
"level": "ERROR",
"service": "api",
"error": map[string]any{
"code": 500,
"message": "Database connection failed",
},
}
toonLog, _ := toon.Encode(logEntry)
fmt.Println(toonLog)config := AppConfig{
Database: DBConfig{Host: "localhost", Port: 5432},
Cache: CacheConfig{TTL: 3600, MaxSize: 1000},
}
toonConfig, _ := toon.Encode(config)
// Compact, readable configuration formatRun the test suite:
go test -vRun benchmarks:
go test -bench=. -benchmem| Example | JSON Tokens | TOON Tokens | Savings |
|---|---|---|---|
| Small product catalog | 117 | 49 | 58.1% |
| API response with users | 123 | 53 | 56.9% |
| Analytics data | 209 | 94 | 55.0% |
| Large dataset (50 records) | 2159 | 762 | 64.7% |
Token counts measured with GPT-4 tokenizer (o200k_base)
JSON (verbose):
{
"items": [
{ "sku": "A1", "name": "Widget", "qty": 2, "price": 9.99 },
{ "sku": "B2", "name": "Gadget", "qty": 1, "price": 14.5 }
]
}TOON (compact):
items[2]{name,price,qty,sku}:
Widget,9.99,2,A1
Gadget,14.5,1,B2
This is a Go implementation of the TOON specification created by Johann Schopplich.
MIT License - see LICENSE for details.
Contributions are welcome! Please feel free to submit issues or pull requests.