What's New

ToJson Protocol

Custom serialization support via the ToJson protocol:

(defrecord Person [first-name last-name]
  ToJson
  (-to-json [_]
    {:name (str first-name " " last-name)}))

(toon/encode (->Person "Alice" "Smith"))
;; => "name: Alice Smith"

Replacer Function

Transform values during encoding with a replacer function:

(toon/encode data {:replacer (fn [k v] (if (= k "password") "***" v))})

Path Expansion with Quoted Keys

Path expansion now correctly tracks and quotes keys that require quoting.

JSON-like Numeric Grammar

• Leading zeros now rejected (007, -007 → treated as strings)
• Scientific notation supported (1e10, 1E-5, 2.5e+3)

Bare List Marker Support

Bare list markers (- alone) now decode to empty objects {}:

[2]:
  -
  - name: Ada

Decodes to: [{} {"name" "Ada"}]

V3.0 Spec: List-Item Tabular Arrays

Enhanced encoding for arrays of objects within list items.

Commits

• feat(encode): add ToJson protocol for custom serialization
• feat(encode): add replacer function for value transformation
• feat(decode): add quoted key tracking for path expansion
• feat(encode): add v3.0 spec list-item tabular array encoding
• fix(decode): enforce JSON-like numeric grammar with scientific notation
• fix(decode): handle bare list marker as empty object

What's New

Streaming Decode API

New event-based streaming decoder for memory-efficient processing of large TOON documents:

• events - Returns lazy sequence of parse events
• events-ch - Returns core.async channel for async processing
• events->value - Reconstructs values from event streams

New API Functions

• lines->value - Decode from pre-split lines
• encode-lines - Streaming line output

Breaking Changes

• Namespace rename: com.vadelabs.toon.interface → com.vadelabs.toon.core
  • Update your requires: [com.vadelabs.toon.core :as toon]

Other Changes

• Applied Stuart Sierra naming conventions
• ClojureScript testing support
• Bug fixes for streaming decode

See CHANGELOG for full details.

TOON v2.0 Compliance & Code Quality Improvements

This release updates the library to comply with TOON v2.0 specification and includes significant code quality improvements through extensive refactoring.

Changed

• TOON v2.0 compliance - Updated from v1.4 to v2.0 specification

• Code quality - Extensive refactoring for better maintainability

  • Extracted helper functions to simplify complex decoders and encoders
  • Reduced function complexity through focused, single-purpose helpers
  • Improved code organization in parser and scanner modules
  • Pre-compiled regex patterns in parser
  • Replaced lazy sequences with eager evaluation in encoders

Added

• Safety improvements

  • Depth limiting to prevent stack overflow in normalize function
  • Validation for empty brackets and negative array lengths in parser
  • Enhanced error messages with specific suggestions

• Test coverage - Improved from 91.5% to 92.6% overall coverage

  • Added tests for max-depth validation and exception metadata
  • Added tests for parser edge cases (empty brackets, negative array lengths)
  • 456 tests with 958 assertions

Installation

;; deps.edn
com.vadelabs/toon {:mvn/version "2025.11.20-10"}

;; Leiningen
[com.vadelabs/toon "2025.11.20-10"]

Links

• Full Changelog
• TOON v2.0 Specification
• Clojars Package

🎉 Key Features

Key Collapsing

• 40-60% token reduction for nested structures
• Collapses nested single-key objects into dotted paths
• Example: {data: {config: {server: "localhost"}}} → data.config.server: localhost
• Safe mode validation with collision detection

Path Expansion

• Reverses key collapsing during decode
• Full lossless round-trip guarantee
• Deep merge support for overlapping paths
• Strict/non-strict conflict resolution

Code Quality

• 33% lower cyclomatic complexity
• 83% less code duplication
• Better performance with pre-compiled regex
• Improved maintainability

📊 Stats

• 435 tests passing (126 new)
• 929 assertions verified
• Zero regressions

⚠️ Breaking Changes

• Removed :length-marker option (always uses [N] format now)

📖 Usage

;; Add to deps.edn
com.vadelabs/toon {:mvn/version "2025.11.12-5"}

;; Key collapsing
(require '[com.vadelabs.toon.interface :as toon])

(toon/encode {:data {:config {:server "localhost"}}} 
             {:key-collapsing :safe})
;=> "data.config.server: localhost"

;; Path expansion
(toon/decode "data.config.server: localhost" 
             {:expand-paths :safe})
;=> {"data" {"config" {"server" "localhost"}}}

🔗 Links

• Full Changelog
• Documentation
• TOON Specification

TOON v2025.11.05-43

First public release! 🎉

A Clojure/ClojureScript implementation of TOON (Token-Oriented Object Notation) - a compact format for passing data to LLMs with significantly fewer tokens than JSON.

Features

• Full TOON v1.3 support - encode and decode between Clojure data and TOON format
• Three array styles - inline for primitives, tabular for uniform objects, list for mixed data
• Flexible options - choose your delimiter (comma, tab, pipe), add length markers, adjust indentation
• Smart string handling - only quotes when necessary, supports Unicode and emoji
• Both platforms - works in Clojure (JVM) and ClojureScript
• Well tested - 340+ tests with 90%+ code coverage including property-based roundtrip testing
• Great errors - helpful messages with suggestions when things go wrong
• Comprehensive docs - README with examples, API reference, and contribution guidelines
• CI/CD - Automated testing and deployment to Clojars via GitHub Actions
• Smart versioning - Version number reflects commits since last release (or all commits for first release)

Why use TOON?

Saves tokens when sending structured data to LLMs:

• 49% fewer tokens than formatted JSON
• 28% fewer than minified JSON
• Works best for uniform arrays of objects (like database query results)

Installation

Clojure CLI/deps.edn

com.vadelabs/toon {:mvn/version "2025.11.05-43"}

Leiningen/Boot

[com.vadelabs/toon "2025.11.05-43"]

Links

• GitHub Repository
• TOON Specification v1.3
• Documentation