STOF: Data that carries its own logic

Standard Transformation and Organization Format

• Docs
• Playground
• GitHub
• Discord
• Install

Overview

Send functions + data over APIs, write configs that validate themselves, build data pipelines where transformations travel with the data, store logic + data in a database, etc.

Works with JSON, YAML, TOML, etc. - no migration needed.

Add/import logic only where required.

Treats everything uniformly - fields, functions, PDFs, images, binaries, etc. - as data that can be combined in a single portable document.

Benefits

• Write data + logic once, use it everywhere (JS, Rust, Python, anywhere your app lives)
• Format-agnostic I/O (works with JSON, YAML, TOML, PDF, binaries, etc.)
• Sandboxed logic + execution in your data (as data)
• Send functions over APIs
• Doesn't need a large ecosystem to work

Example Use-Cases

• Smart configs with validation and logic
• Data interchange with sandboxed execution
• Prompts as human-readable & maintainable data + code
• AI/LLM workflows and model configs
• Data pipelines with built-in processing
• Integration glue between systems
• Self-describing datasets
• ... basically anywhere data meets logic

Sample Stof

Check out the online playground for examples you can play with yourself.

#[attributes("optional exec control | metadata | meta-logic")]
// A field on the doc "root" node.
field: 42

// JSON-like data & function organization
stats: {
    // Optional field types & expressions
    prompt context: prompt("trees of strings", tag="optional-xml-tag",
        prompt("behaves like a tree for workflows & functions"),
        prompt("just cast to/from str anywhere strings are needed")
        // Std.prompt(..) can take N prompts as sub-prompts
    );
    
    // Units as types with conversions & casting
    cm height: 6ft + 2in
    MiB memory: 2MB + 50GiB - 5GB + 1TB
    ms ttl: 300s
}

#[main]
/// The CLI (and other envs) use the #[main] attribute for which fns to call on run.
fn do_something() {
    // Dot separated path navigation of the document (self is the current node/obj)
    let gone = self.self_destruction();
    assert(gone);

    // async functions, blocks, and expressions always available
    async {
        const now = Time.now();
        loop {
            sleep(20ms);
            if (Time.diff(now) > 2s) break;
        }
    }

    // partial I/O with any format
    pln(stringify("toml", self.stats));
}

/**
 * A function that removes itself from this document when executed.
 */
fn self_destruction() -> bool {
    pln(self.field); // Std.pln(..) print line function
    drop(this);      // "this" is always the last fn on the call stack
    true             // "return" keyword is optional (no ";")
}

CLI

See installation docs for CLI instructions and more information.

#[main]
fn say_hi() {
    pln("Hello, world!");
}

> stof run example.stof
Hello, world!

Embedded Stof

Stof is written in Rust, and is meant to be used wherever you work. Join the project Discord to get involved.

Rust

[dependencies]
stof = "0.8.*"

use stof::model::Graph;

fn main() {
    let mut graph = Graph::default();
    
    graph.parse_stof_src(r#"
        #[main]
        fn main() {
            pln("Hello, world!");
        }
    "#, None).unwrap();

    match graph.run(None, true) {
        Ok(res) => println!("{res}"),
        Err(err) => panic!("{err}"),
    }
}

Python

Stof is available on PyPi. Just pip install stof and import the pystof module to get started.

A few examples are located in the src/py/examples folder.

from pystof import Doc

STOF = """
#[main]
fn main() {
    const name = Example.name('Stof,', 'with Python');
    pln(`Hello, ${name}!!`)
}
"""

def name(first, last):
    return first + ' ' + last

def main():
    doc = Doc()
    doc.lib('Example', 'name', name)
    doc.parse(STOF)
    doc.run()

if __name__ == "__main__":
    main()

# Output:
# Hello, Stof, with Python!!

JavaScript/TypeScript

Stof is compiled to WebAssembly for embedding in JS, and a JSR package is provided.

A few examples are located in the web/examples folder.

import { StofDoc } from '@formata/stof';
const doc = await StofDoc.new();

doc.lib('Std', 'pln', (... vars: unknown[]) => console.log(...vars));
doc.lib('Example', 'nested', async (): Promise<Map<string, string>> => {
    const res = new Map();
    res.set('msg', 'hello, there');
    res.set('nested', await (async (): Promise<string> => 'this is a nested async JS fn (like fetch)')());
    return res;
}, true);

doc.parse(`
    field: 42
    fn main() -> int {
        const res = await Example.nested();
        pln(res);
        self.field
    }
`);
const field = await doc.call('main');
console.log(field);

/*
Map(2) {                                                                                                                                                                                                                       
  "msg" => "hello, there",
  "nested" => "this is a nested async JS fn (like fetch)"
}
42
*/

Research & Exploration

Stof explores several research areas:

• Practical code mobility at scale with modern type systems
• Security models for distributed computation-as-data
• Performance characteristics of serializable computation vs traditional RPC
• Formal semantics for "code as data" in distributed systems
• Edge computing, data pipelines, and collaborative systems

License

Apache 2.0. See LICENSE for details.

Feedback & Community

• Open issues or discussions on GitHub
• Chat with us on Discord
• Star the project to support future development!

Reach out to [email protected] to contact us directly