PJS - Priority JSON Streaming Protocol

Priority-based streaming | Progressive loading | Zero-copy operations | WebAssembly ready

High-performance Rust library for priority-based JSON streaming with SIMD acceleration. Stream large JSON responses progressively, delivering critical data first while background data loads asynchronously.

Important

GAT migration (1.82x faster), HTTP adapter with CQRS, comprehensive security hardening with bounded iteration and input validation. Requires nightly Rust for zero-cost async abstractions.

Features

• Blazing Fast - SIMD-accelerated parsing, GAT-based zero-cost abstractions (1.82x faster than async_trait)
• Smart Streaming - Priority-based delivery sends critical data first, skeleton-first rendering
• Memory Efficient - Optimized progressive loading, bounded memory usage, zero-copy operations
• WebAssembly - Browser and Node.js support with compact bundle (~70KB gzipped)
• Secure - Defense-in-depth decompression protection, DoS prevention, input validation
• Schema Aware - Automatic compression and semantic analysis
• Production Ready - Clean Architecture, comprehensive test suite, Prometheus metrics

Performance

Benchmark	Performance Gain	Notes
GAT Async	1.82x faster	Static dispatch eliminates virtual calls
Small JSON	Competitive	Comparable to industry standards
Medium JSON	~3x faster	vs traditional parsers
Large JSON	~6x faster	vs traditional parsers
Progressive Loading	~5x faster	vs batch processing

Installation

cargo add pjson-rs

Or add to Cargo.toml:

[dependencies]
pjson-rs = "0.4"

Note

Requires Rust 1.89+ (nightly). See MSRV policy for details.

Quick Start

Rust HTTP Server

use pjson_rs::infrastructure::http::axum_adapter::create_pjs_router;

#[tokio::main]
async fn main() {
    let app = create_pjs_router().with_state(app_state);
    let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await?;
    axum::serve(listener, app).await?;
}

Start server and test:

# Run example server
cargo run --example axum_server

# Create session
curl -X POST http://localhost:3000/pjs/sessions \
  -H "Content-Type: application/json" \
  -d '{"max_concurrent_streams": 5}'

# Stream data with priority
curl http://localhost:3000/pjs/stream/SESSION_ID/sse

WebAssembly (Browser)

npm install pjs-wasm

PriorityStream API (Recommended)

<script type="module">
import init, { PriorityStream, PriorityConstants } from './pkg/pjs_wasm.js';

async function main() {
    await init();

    const stream = new PriorityStream();
    stream.setMinPriority(PriorityConstants.MEDIUM());

    // Register callbacks
    stream.onFrame((frame) => {
        console.log(`${frame.type} [${frame.priority}]: ${frame.payload}`);
        if (frame.priority >= 80) {
            updateUI(JSON.parse(frame.payload)); // High priority first
        }
    });

    stream.onComplete((stats) => {
        console.log(`Completed: ${stats.totalFrames} frames in ${stats.durationMs}ms`);
    });

    // Start streaming
    stream.start(JSON.stringify({ id: 123, name: "Alice", bio: "..." }));
}

main();
</script>

Tip

Use the PriorityStream API for automatic frame handling and built-in security limits. Ideal for real-time dashboards and progressive loading.

Simple Parser API

<script type="module">
import init, { PjsParser, PriorityConstants } from './pkg/pjs_wasm.js';

async function main() {
    await init();
    const parser = new PjsParser();

    const frames = parser.generateFrames(
        JSON.stringify({ user_id: 123, name: "Alice" }),
        PriorityConstants.MEDIUM()
    );

    frames.forEach(frame => {
        if (frame.priority >= 90) {
            updateUI(frame.data); // Critical data first
        }
    });
}

main();
</script>

Interactive Demo

Try the Browser Demo with transport switching, performance benchmarks, and real-time metrics.

WebAssembly (Node.js)

import init, { PjsParser } from 'pjs-wasm';
import { readFile } from 'fs/promises';

const wasmBuffer = await readFile('./node_modules/pjs-wasm/pkg/pjs_wasm_bg.wasm');
await init(wasmBuffer);

const parser = new PjsParser();
const frames = parser.generateFrames(JSON.stringify(data), 50);

frames.forEach(frame => {
    console.log(`Priority ${frame.priority}: ${frame.frame_type}`);
});

Build WASM from Source

# Install wasm-pack
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

# Build for different targets
wasm-pack build crates/pjs-wasm --target web --release      # Browsers
wasm-pack build crates/pjs-wasm --target nodejs --release   # Node.js
wasm-pack build crates/pjs-wasm --target bundler --release  # Webpack/Rollup

Use Cases

• Real-time Dashboards - Show key metrics instantly, load details progressively
• Mobile Apps - Optimize for slow networks, critical data first
• E-commerce - Product essentials load immediately, reviews/images follow
• Financial Platforms - Trading data prioritized over historical charts
• Gaming Leaderboards - Player rank appears instantly, full list streams in

Building

Prerequisites

Warning

This project requires nightly Rust for Generic Associated Types (GAT) features. Stable Rust is not supported.

rustup install nightly
rustup override set nightly

Build Commands

# Standard build
cargo build --release

# Run tests with nextest
cargo nextest run --workspace

# Run benchmarks
cargo bench -p pjs-bench

# Run demo servers
cargo run --bin websocket_streaming --manifest-path crates/pjs-demo/Cargo.toml
cargo run --bin interactive_demo --manifest-path crates/pjs-demo/Cargo.toml

Feature Flags

Tip

Start with default features. Add extras only when needed to keep compile times fast.

Feature	Description	Default
simd-auto	Auto-detect SIMD support	✅ Yes
simd-avx2	Force AVX2 SIMD	No
simd-neon	Force ARM NEON	No
schema-validation	Schema validation engine	✅ Yes
compression	Schema-based compression	No
http-server	Axum HTTP server	No
websocket-server	WebSocket streaming	No
prometheus-metrics	Prometheus integration	No
jemalloc	Use jemalloc allocator	No
mimalloc	Use mimalloc allocator	No

Security

PJS includes built-in security features to prevent DoS attacks:

import { PriorityStream, SecurityConfig } from 'pjs-wasm';

const security = new SecurityConfig()
    .setMaxJsonSize(5 * 1024 * 1024)  // 5 MB limit
    .setMaxDepth(32);                  // 32 levels max

const stream = PriorityStream.withSecurityConfig(security);

Default Limits:

• Max JSON size: 10 MB
• Max nesting depth: 64 levels
• Max array elements: 10,000
• Max object keys: 10,000

Important

Security: Comprehensive multi-layer protection including bounded iteration (DoS prevention), input validation, and 4-layer defense-in-depth decompression protection against compression bombs (CVSS 7.5 vulnerabilities fixed).

Decompression Security:

• MAX_RLE_COUNT: 100,000 items per run
• MAX_DELTA_ARRAY_SIZE: 1,000,000 elements
• MAX_DECOMPRESSED_SIZE: 10 MB total
• Integer overflow protection: Checked arithmetic throughout

Architecture

PJS follows Clean Architecture with Domain-Driven Design:

• pjs-domain - Pure business logic, WASM-compatible
• pjs-wasm - WebAssembly bindings with PriorityStream API, security limits
• pjs-core - Rust implementation with HTTP/WebSocket integration
• pjs-demo - Interactive demo servers with real-time streaming
• pjs-js-client - TypeScript/JavaScript client with WasmBackend transport
• pjs-bench - Comprehensive performance benchmarks

Key Features:

• GAT Migration: Zero-cost async abstractions (1.82x faster than async_trait)
• HTTP Adapter: 8 REST endpoints with CQRS pattern
• Security Hardening: Bounded iteration, input validation, decompression bomb protection
• Generic Type System: Type-safe Id wrappers, generic InMemoryStore<K, V>
• Platform Support: Windows, Linux, macOS validated

Contributing

Contributions welcome! Please ensure:

rustup override set nightly
cargo clippy --workspace -- -D warnings
cargo nextest run --workspace --all-features
cargo +nightly fmt --check

See CONTRIBUTING.md for guidelines.

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT License (LICENSE-MIT)

at your option.

Resources

• Protocol Specification
• Changelog
• Benchmarks
• Discussions

---

PJS: Priority-based JSON streaming for instant user experiences.