Crate serde_toon 

serde_toon

A Serde-compatible serialization library for the TOON (Token-Oriented Object Notation) format.

What is TOON?

TOON is a compact, human-readable data format specifically designed for efficient communication with Large Language Models (LLMs). It achieves 30-60% fewer tokens than equivalent JSON while maintaining readability and structure.

Key Features

• Token-Efficient: Minimalist syntax reduces token count by eliminating unnecessary braces, brackets, and quotes
• Tabular Arrays: Homogeneous object arrays serialize as compact tables with headers
• Serde Compatible: Works seamlessly with existing Rust types via #[derive(Serialize, Deserialize)]
• Type Safe: Statically typed with comprehensive error reporting
• No Unsafe Code: Written entirely in safe Rust with zero unsafe blocks

Quick Start

Add this to your Cargo.toml:

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_toon = "0.2"

Basic Serialization and Deserialization

use serde::{Deserialize, Serialize};
use serde_toon::{to_string, from_str};

#[derive(Serialize, Deserialize, PartialEq, Debug)]
struct User {
    id: u32,
    name: String,
    active: bool,
}

let user = User {
    id: 123,
    name: "Alice".to_string(),
    active: true,
};

// Serialize to TOON format
let toon_string = to_string(&user).unwrap();
// Output: "id: 123\nname: Alice\nactive: true"

// Deserialize back
let user_back: User = from_str(&toon_string).unwrap();
assert_eq!(user, user_back);

Working with Arrays (Tabular Format)

Arrays of homogeneous objects automatically serialize as space-efficient tables:

use serde::{Deserialize, Serialize};
use serde_toon::to_string;

#[derive(Serialize, Deserialize)]
struct Product {
    id: u32,
    name: String,
    price: f64,
}

let products = vec![
    Product { id: 1, name: "Widget".to_string(), price: 9.99 },
    Product { id: 2, name: "Gadget".to_string(), price: 14.99 },
];

let toon = to_string(&products).unwrap();
// Output: "[2]{id,name,price}:\n  1,Widget,9.99\n  2,Gadget,14.99"

Dynamic Values with toon! Macro

use serde_toon::{toon, Value};

let data = toon!({
    "name": "Alice",
    "age": 30,
    "tags": ["rust", "serde", "llm"]
});

if let Value::Object(obj) = data {
    assert_eq!(obj.get("name").and_then(|v| v.as_str()), Some("Alice"));
}

Performance Characteristics

• Serialization: O(n) where n is the number of fields/elements
• Deserialization: O(n) with single-pass parsing
• Memory: Pre-allocated buffers minimize reallocations
• Token Count: 30-60% reduction vs JSON for typical structured data

Safety Guarantees

• No unsafe code blocks
• All array indexing is bounds-checked
• Proper error propagation with Result types
• No panics in public API (except for logic errors that indicate bugs)

Format Specification

For the complete TOON format specification, see the spec module documentation.

External reference: https://github.com/johannschopplich/toon

Examples

See the examples/ directory for focused, production-ready examples:

• simple.rs - Your first TOON experience (basic serialization)
• macro.rs - Building values with the toon! macro
• tabular_arrays.rs - TOON’s tabular feature for repeated structures
• dynamic_values.rs - Working with Value dynamically
• custom_options.rs - Customizing delimiters and formatting
• token_efficiency.rs - TOON vs JSON comparison

Run any example with: cargo run --example <name>

Re-exports

pub use de::Deserializer;

pub use error::Error;

pub use error::Result;

pub use map::ToonMap;

pub use options::Delimiter;

pub use options::ToonOptions;

pub use ser::Serializer;

pub use ser::ValueSerializer;

pub use value::Number;

pub use value::Value;

Modules

de

TOON deserialization.

error

Error types for TOON serialization and deserialization.

macros

map

Ordered map type for TOON objects.

options

Configuration options for TOON serialization.

ser

TOON serialization.

spec

TOON Format Specification

value

Dynamic value representation for TOON data.

Macros

toon

Functions

from_reader

Deserialize an instance of type T from an I/O stream of TOON.

from_slice

Deserialize an instance of type T from bytes of TOON text.

from_str

Deserialize an instance of type T from a string of TOON text.

to_string

Serialize any T: Serialize to a TOON string.

to_string_pretty

Serialize any T: Serialize to a pretty-printed TOON string.

to_string_with_options

Serialize any T: Serialize to a TOON string with custom options.

to_value

Convert any T: Serialize to a Value.

to_writer

Serialize any T: Serialize to a writer in TOON format.

to_writer_with_options

Serialize any T: Serialize to a writer in TOON format with custom options.