Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Shellshock9001/typeshock

Repository files navigation

TypeShock — Constrained Type Generation

TypeShock

Constrained LLM output generation using TypeScript interfaces as the contract.
The LLM never generates structure — only fills in values. Every field, every type, every enum: enforced.

InstallationWeb UIPython LibraryHow It WorksType SupportValidationExportLimitations

Python License Version Tests


What Is TypeShock?

TypeShock is a constrained decoding engine that guarantees LLM output conforms to TypeScript interface definitions. Instead of asking an LLM to "please output valid JSON" and hoping for the best, TypeShock forces structural compliance at the token level.

It ships with two interfaces:

  1. Python Library — Drop-in constrained decoder for any HuggingFace causal LM (LLaMA, Mistral, Phi, etc.). The LLM only generates leaf values (strings, numbers, booleans); all structural tokens ({, }, [, ], :, ,) are produced deterministically by the engine.

  2. Web UI — A full-featured browser application with a TypeScript editor, natural language prompt bar, real-time validation trace, multi-format export, and generation history. Powered by a FastAPI backend that calls the Mistral API for generation and runs the TypeShock parser + validator locally.

Inspired by jsonformer by Ramp. TypeShock extends the concept with TypeScript-native input, enum/literal union constraints, nullable types, optional fields, tuple support, and a production web UI.


Why TypeShock?

If your codebase is TypeScript, you already have your data contracts written — they're your interfaces. But the moment you want an LLM to generate data conforming to those contracts, you're told to rewrite everything as JSON Schema, Pydantic models, or Zod validators. That's duplicate work, and it drifts.

TypeShock skips that entirely. You paste the same interface your frontend team already uses, and the engine guarantees the LLM's output satisfies it — field names, types, enum constraints, nullable rules, nesting depth, all of it. No translation layer. No schema drift. No "the LLM sometimes forgets a field."

This matters most when you're working with production API types. A Stripe PaymentIntent has 7 possible status values, nullable descriptions, and 3 levels of nested address objects. A Kubernetes V1Pod goes 6 levels deep with enum-constrained protocols and restart policies. TypeShock's parser and test suite are validated against these exact schemas — not toy examples. If your interfaces compile in tsc, they'll work here.

The alternative is prompt engineering: "Please output valid JSON matching this schema, and make sure status is one of these 7 values, and description can be null but not missing, and..." — and then writing a validator to catch when it doesn't. TypeShock replaces both the prompt engineering and the validator. The structure is enforced at generation time, not after.


Installation

Prerequisites

Requirement Version Purpose
Python 3.9+ Runtime
pip or poetry any Package management
Git any Clone the repo
Node.js Not required

For the Python library (local model inference), you also need:

  • PyTorch 2.0+
  • HuggingFace Transformers 4.30+
  • A GPU with ≥4GB VRAM (or CPU with patience)

For the Web UI, you also need:

Step 1: Clone the Repository

git clone [email protected]:Shellshock9001/typeshock.git
cd typeshock

Step 2: Install Dependencies

Option A — Poetry (recommended):

pip install poetry
poetry install

Option B — pip:

pip install -e .

Option C — Web UI only (no local models):

pip install fastapi uvicorn httpx pyyaml termcolor

Step 3: Set Your Mistral API Key

⚠️ Required for the Web UI. The Web UI calls the Mistral API to generate data. You must provide your own API key — no key is included in the repo.

Get a free API key from console.mistral.ai, then set it as an environment variable:

# Linux / macOS
export MISTRAL_API_KEY="your-api-key-here"

# Windows PowerShell
$env:MISTRAL_API_KEY="your-api-key-here"

# Windows CMD
set MISTRAL_API_KEY=your-api-key-here

To make this permanent, add the export line to your ~/.bashrc, ~/.zshrc, or use Windows System Environment Variables.

Note: The Python library (constrained decoding with local models) does NOT need this key — it runs entirely offline using HuggingFace models. The API key is only required for the Web UI's Mistral-powered generation backend.

Step 4: Run the Web UI

python web/server.py

Open http://localhost:3847 in your browser.


Web UI

The TypeShock web interface is a dark-themed, glassmorphic application with three main panels:

TypeShock Editor

Left Panel — TypeScript Interface Editor

Write or paste any TypeScript interface definition. The editor accepts:

  • Single interfaces (interface Product { ... })
  • Multiple interfaces with cross-references (interface Address { ... } then address: Address in another interface)
  • Enums (enum Color { Red = "red", ... })
  • Type aliases (type Status = "active" | "inactive")
  • export and readonly modifiers (parsed and ignored gracefully)

A schema analysis bar sits below the editor showing parsed metadata badges:

  • Field type distribution (e.g., string 5, number 3, enum 1, boolean 1, array 2)
  • Nesting depth
  • Total field count

Right Panel — Output Viewer

After generation, the output is displayed with syntax highlighting. Six format tabs are available along the top:

Tab Output File Extension
.json Raw JSON with indentation .json
.ts Full TypeScript file with interface + typed const export .ts
.yaml YAML serialization .yaml
.py Python dict with type annotations .py
.csv Flat CSV (headers + values) .csv
.schema The parsed JSON Schema used for validation .schema.json

Each format produces a real, valid file — not just text with a different extension. The .ts output includes the original interface definition and a typed as const export. The .py output includes from typing import ... imports and proper Dict[str, Any] annotations.

Download uses the browser's native showSaveFilePicker() API (the same API used by Figma, VS Code Web, and Excalidraw). When you click the download button, your OS "Save As" dialog opens and you choose exactly where to save the file. If the browser doesn't support this API, it falls back to a standard anchor-based download.

Bottom Bar — Prompt Input

A natural language prompt describing what data to generate. The engine uses intelligent multi-item detection: if your prompt contains phrases like "and", "both", "two", "multiple", "list of", or "various", it automatically switches to multi-item mode and generates an array of distinct objects.

Examples:

  • Generate a premium wireless headphone product listing → 1 item
  • Generate a premium wireless headphone product and an iPhone product → 2 items with distinct data
  • Generate a list of 5 different electronics products → 5 items

Sidebar — Generation History

Every generation is saved to an in-memory history log with:

  • Timestamp
  • Prompt snippet
  • Validation verdict (PASS / FAIL badge)
  • Duration in milliseconds

Click any history entry to reload that generation. "Clear History" wipes all entries.

Validation Trace Panel

Below the output, a contract enforcement panel shows exactly how the output was validated:

TypeShock Validation

Header bar:

  • Verdict badge: PASS (green) or FAIL (red)
  • Fields parsed count (e.g., 30 fields parsed)
  • Satisfaction ratio (e.g., 30/30 satisfied)
  • Violation count (e.g., 0 violations)
  • Percentage (e.g., 100%)
  • Multi-item badge when applicable (e.g., 2 items)

Per-field trace rows: Each field gets its own row showing:

  • ✅ or ❌ pass/fail icon
  • Field name and JSON path (e.g., name $.name)
  • Type badge (string, number, boolean, enum, array, object)
  • Actual value preview (e.g., String (31 chars), Valid number: 349.99, Boolean: True, Array with 8 items, 8/8 items valid, Value 'electronics' is in allowed set)

Expandable array/object rows: Click the arrow on array or object fields to expand and see per-element or per-property validation.

Action buttons:

  • Retry — Re-run generation with the same schema and prompt
  • Violations — Filter to show only failed checks
  • Diff — Compare two generations side-by-side (when history exists)

Python Library

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from typeshock import TypeShock

# Load any HuggingFace causal LM
model = AutoModelForCausalLM.from_pretrained("microsoft/phi-2", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-2")

# Define your schema as a TypeScript interface
ts = TypeShock(
    model=model,
    tokenizer=tokenizer,
    typescript_schema="""
        interface JobCandidate {
            name: string;
            title: string;
            yearsExperience: number;
            skills: string[];
            currentlyEmployed: boolean;
            preferredRole: "backend" | "frontend" | "fullstack" | "ml-engineer";
            education: {
                degree: string;
                university: string;
                year: number;
            };
            openToRelocation?: boolean;
        }
    """,
    prompt="Generate a profile for a senior ML engineer",
    debug=True,
    temperature=0.7,
)

result = ts()
# Returns a dict GUARANTEED to match the interface:
# {
#   "name": "Alice Chen",
#   "title": "Senior ML Engineer",
#   "yearsExperience": 8,
#   "skills": ["PyTorch", "Kubernetes", "MLOps"],
#   "currentlyEmployed": true,
#   "preferredRole": "ml-engineer",  ← constrained to enum values
#   "education": {
#     "degree": "MS Computer Science",
#     "university": "Stanford",
#     "year": 2016
#   }
#   // openToRelocation may or may not be present (optional)
# }

Real-World API Contract: Stripe PaymentIntent

The library handles the same deeply nested, enum-heavy interfaces you'd find in production SDKs. Here's a Stripe PaymentIntent — the kind of schema where prompt-based JSON generation breaks down:

ts = TypeShock(
    model=model,
    tokenizer=tokenizer,
    typescript_schema="""
        interface PaymentIntent {
            id: string;
            object: "payment_intent";
            amount: number;
            currency: string;
            status: "requires_payment_method" | "requires_confirmation"
                  | "requires_action" | "processing" | "requires_capture"
                  | "canceled" | "succeeded";
            capture_method: "automatic" | "automatic_async" | "manual";
            description: string | null;
            livemode: boolean;
            metadata: Record<string, string>;
            payment_method_types: string[];
            last_payment_error?: {
                code: string;
                message: string;
                type: "api_error" | "card_error" | "idempotency_error"
                    | "invalid_request_error";
            };
            shipping?: {
                address: {
                    city: string | null;
                    country: string | null;
                    line1: string | null;
                    line2: string | null;
                    postal_code: string | null;
                    state: string | null;
                };
                name: string;
                phone: string | null;
            };
        }
    """,
    prompt="Generate a succeeded payment for a $249.99 mechanical keyboard",
)

result = ts()
# Every field is type-correct:
#   - object is ALWAYS "payment_intent" (literal enforcement)
#   - status is ALWAYS one of 7 values (enum constraint)
#   - description is either a string or null (nullable)
#   - last_payment_error may or may not be present (optional)
#   - shipping.address fields are all nullable strings
#   - metadata is a Record<string, string> (3 key-value pairs)
#
# An unconstrained LLM would hallucinate status values, forget nullable
# fields, or nest objects incorrectly. TypeShock makes that impossible.

Pretty Printing

from typeshock.format import print_highlighted

print_highlighted(result)
# Prints with ANSI colors:
# - Keys in cyan
# - String values in green
# - Numbers in green
# - Null values in yellow
# - Structure ({, }, [, ]) in default color

Parser Standalone

You can use the TypeScript parser independently:

from typeshock import parse_typescript_interface

schema = parse_typescript_interface("""
    enum Color { Red = "red", Green = "green", Blue = "blue" }

    interface Product {
        id: string;
        name: string;
        price: number;
        color: Color;
        tags: string[];
        metadata?: Record<string, string>;
    }
""")

# Returns JSON Schema-compatible dict:
# {
#   "type": "object",
#   "properties": {
#     "id": {"type": "string"},
#     "name": {"type": "string"},
#     "price": {"type": "number"},
#     "color": {"type": "enum", "values": ["red", "green", "blue"]},
#     "tags": {"type": "array", "items": {"type": "string"}},
#     "metadata": {"type": "object", "additionalProperties": {"type": "string"}, "optional": true}
#   },
#   "required": ["id", "name", "price", "color", "tags"]
# }

Configuration Parameters

Parameter Type Default Description
model PreTrainedModel required Any HuggingFace causal language model
tokenizer PreTrainedTokenizer required Matching tokenizer for the model
typescript_schema str or dict required TypeScript interface definition (or pre-parsed JSON Schema dict)
prompt str required Natural language description of what to generate
debug bool False Print colored generation progress to stdout
max_array_length int 10 Maximum elements per array (LLM decides actual length via logit comparison)
max_number_tokens int 6 Maximum tokens for a single number value
temperature float 1.0 Sampling temperature (higher = more random)
max_string_token_length int 10 Maximum tokens for a single string value

How It Works

TypeShock uses constrained decoding — a technique where the LLM's vocabulary is restricted at each generation step so it can only produce tokens that lead to valid output.

Architecture

┌─────────────────┐     ┌──────────────┐     ┌─────────────────┐
│  TypeScript      │────▶│  Parser      │────▶│  JSON Schema    │
│  Interface       │     │  (Recursive  │     │  (Internal      │
│  Definition      │     │   Descent)   │     │   Repr)         │
└─────────────────┘     └──────────────┘     └────────┬────────┘
                                                       │
                                                       ▼
┌─────────────────┐     ┌──────────────┐     ┌─────────────────┐
│  Structured      │◀────│  Schema      │◀────│  Constrained    │
│  Output          │     │  Walker      │     │  Generators     │
│  (Guaranteed)    │     │  (Recursive) │     │  (Per-Type)     │
└─────────────────┘     └──────────────┘     └─────────────────┘
                                                       │
                                              ┌────────┴────────┐
                                              │                 │
                                         ┌────▼────┐     ┌─────▼─────┐
                                         │  Logits  │     │  Stopping  │
                                         │  Warpers │     │  Criteria  │
                                         └─────────┘     └───────────┘

Step-by-Step Process

  1. Parse — The TypeScript interface is tokenized and parsed by a recursive descent parser into a JSON Schema-compatible internal representation. Supports interface, enum, type alias, export, readonly, comments, and all type constructs listed below.

  2. Walk — The schema walker iterates over each property in the schema. For each field, it places a GENERATION_MARKER placeholder in the partial output.

  3. Prompt — The partial output (with marker) is serialized as JSON and appended to the user's prompt along with the TypeScript definition. This context tells the LLM what has been generated so far and what comes next.

  4. Constrain — Based on the field type, a specific constrained generator is invoked:

    Type Constraint Method
    string Generate until closing " is produced (StringStoppingCriteria)
    number Vocabulary masked to digits + . + - only (OutputNumbersTokens), stops on whitespace (NumberStoppingCriteria)
    boolean Single forward pass, compare logits for "true" vs "false" tokens
    enum Vocabulary masked to valid prefix tokens (OutputEnumTokens), stops on exact match (EnumStoppingCriteria)
    array Generate elements, after each check logits for , (continue) vs ] (stop)
    object Recurse into properties
    tuple Fixed-length array with per-position schema
    null Return None directly
  5. Replace — The marker is replaced with the generated value, and the walker moves to the next field.

  6. Return — After all fields are filled, the complete dict is returned. It is structurally guaranteed to match the interface.


Type Support

Fully Supported Types

TypeScript Syntax Internal Representation Example
string {"type": "string"} name: string
number {"type": "number"} price: number
boolean {"type": "boolean"} active: boolean
string[] {"type": "array", "items": {"type": "string"}} tags: string[]
Array<T> {"type": "array", "items": ...} items: Array<number>
T[][] Nested array matrix: string[][]
"a" | "b" | "c" {"type": "enum", "values": [...]} status: "active" | "inactive"
enum Name { ... } {"type": "enum", "values": [...]} enum Color { Red = "red" }
type Alias = "a" | "b" Stored as enum, resolved on reference type Dir = "n" | "s"
T | null {"type": "T", "nullable": true} bio: string | null
field?: T {"type": "T", "optional": true} nickname?: string
{ nested: T } {"type": "object", "properties": {...}} Inline object types
[T, U] {"type": "tuple", "items": [...]} coords: [number, number]
Record<string, T> {"type": "object", "additionalProperties": ...} metadata: Record<string, string>
export interface Parsed normally (export is skipped) export interface User { }
readonly field Parsed normally (readonly is skipped) readonly id: string
Interface references Resolved inline address: Address
// comments Stripped during tokenization Single-line and /* block */

What It Cannot Do

Feature Status Reason
Generic interfaces (interface Box<T>) ❌ Not supported Requires type parameter resolution at parse time
Intersection types (A & B) ❌ Not supported Would need structural merging
Mapped types ({ [K in keyof T]: ... }) ❌ Not supported Requires full type-level computation
Conditional types (T extends U ? X : Y) ❌ Not supported Turing-complete type system feature
extends (interface inheritance) ⚠️ Parsed but fields not merged Parent interface fields are skipped
Index signatures ([key: string]: T) ❌ Not supported Use Record<string, T> instead
Template literal types (`${A}-${B}`) ❌ Not supported String-level type computation
unknown, any, never, void ❌ Not supported No meaningful constrained generation possible
Discriminated unions ⚠️ Partial Picks first non-null variant; no tag-based dispatch

Validation Engine

The validator (web/validator.py) performs deep, recursive contract enforcement against the parsed JSON Schema. It is not a simple "is this valid JSON?" check — it validates every field individually and produces a per-field trace.

What Gets Validated

Check Description
Type match Is the value's runtime type correct? (stringstr, numberint/float, etc.)
Enum compliance Is the value one of the allowed enum values?
Nullable handling If the field is T | null, is null acceptable?
Array structure Is it actually an array? Are all elements valid against the item schema?
Object structure Are all required properties present? Are nested properties valid?
Required fields Are all non-optional fields present in the output?
Nested recursion Validates to arbitrary depth (tested to 10 levels)

Multi-Item Validation

When the output contains {"items": [...]} (from multi-item generation), the validator automatically detects this and validates each item individually against the schema. The trace shows per-item results and aggregates the totals.


Export Formats

Format Content Use Case
JSON Pretty-printed JSON with 2-space indentation APIs, data interchange
TypeScript Interface definition + typed const with as const assertion Frontend codebases, type-safe data
YAML Human-readable YAML serialization Configuration files, Kubernetes manifests
Python Typed dict with from typing import ... imports Python projects, data fixtures
CSV Headers + flat values (nested objects serialized as JSON strings) Spreadsheets, data analysis
JSON Schema The parsed schema used for validation Documentation, schema registries

Testing

The test suite includes 60+ tests across three test files:

# Run all tests
pytest tests/ -v

# Run only parser tests
pytest tests/test_parser.py -v

# Run production schema tests (Stripe, GitHub, OpenAI, Kubernetes, Shopify, Datadog)
pytest tests/test_production_schemas.py -v

Test Coverage

Test File What It Tests # Tests
test_parser.py Tokenizer, primitives, optionals, arrays, nested objects, unions, enums, tuples, edge cases 25+
test_production_schemas.py Real-world schemas from Stripe, GitHub, OpenAI, Kubernetes, Shopify, Datadog 15+
test_init.py Package imports and version 2

Production schema tests are derived from actual TypeScript SDK type definitions:

  • StripePaymentIntent (7-value status enum, nullable description, nested shipping address with 6 nullable fields), Charge (billing details with 4-level nesting, fraud details with nullable union)
  • GitHubPullRequest (30+ fields, 5-level nesting, arrays of labeled objects with nullable descriptions, requested reviewers), WebhookEvent (11-value action enum, nested repository with nullable language)
  • OpenAIChatCompletion (choices array with nested message/tool_calls/function, 5-value finish_reason enum, usage with optional token detail breakdowns), CreateEmbeddingResponse (array of number arrays)
  • KubernetesV1Pod (the deepest nesting test — containers with ports/env/resources/probes, 6+ levels deep, multiple enum fields: TCP/UDP/SCTP, Always/IfNotPresent/Never, etc.)
  • ShopifyProduct (variants with selectedOptions, images with nullable altText, weight units enum, nested SEO)
  • DatadogMonitor (9-value type enum including values with spaces like "event alert", 7-value overall_state, nested thresholds with optional warning/recovery)
  • Stress tests — 10-level deep nesting, 20-value enums, arrays of objects containing arrays, mixed optional + nullable fields

Project Structure

typeshock/
├── typeshock/                  # Core Python library
│   ├── __init__.py             # Package entry point, exports TypeShock + parse_typescript_interface
│   ├── main.py                 # TypeShock class — constrained decoding engine (644 lines)
│   ├── parser.py               # Recursive descent TypeScript parser (529 lines)
│   ├── logits_processors.py    # Custom HuggingFace LogitsWarpers + StoppingCriteria (228 lines)
│   └── format.py               # ANSI color pretty printer for generated output (102 lines)
│
├── web/                        # Web UI (FastAPI + vanilla JS)
│   ├── server.py               # FastAPI backend — parse, generate, validate, download endpoints
│   ├── validator.py            # Deep schema contract enforcement engine (440 lines)
│   └── static/
│       ├── index.html          # Single-page app shell
│       ├── style.css           # Dark-themed glassmorphic design system (21KB)
│       └── app.js              # Client-side logic — editor, output, validation, history (24KB)
│
├── tests/
│   ├── test_parser.py          # 25+ parser unit tests
│   ├── test_production_schemas.py  # 15+ real-world API schema tests
│   └── test_init.py            # Package smoke tests
│
├── examples/
│   └── basic_usage.py          # Complete example with model loading + generation
│
├── docs/                       # Screenshots for README
├── output/                     # Downloaded files land here (gitignored)
├── pyproject.toml              # Poetry project config
├── LICENSE                     # MIT License
└── .gitignore

API Endpoints (Web Server)

Method Path Description
POST /api/parse Parse TypeScript → JSON Schema
POST /api/generate Generate structured data (calls Mistral API)
POST /api/validate Validate arbitrary data against a TypeScript schema
POST /api/analyze Analyze schema structure (returns field metadata + badges)
POST /api/download/{fmt} Convert output to file format and trigger native Save As
GET /api/history Get all generation history
DELETE /api/history/{id} Delete a specific history entry
DELETE /api/history Clear all history

Limitations

Parser Limitations

  • Does not support TypeScript generics (interface Box<T> { value: T })
  • Does not merge extends parent fields (the keyword is parsed but parent fields are skipped)
  • Intersection types (A & B) are not supported
  • Mapped types, conditional types, template literals, and other advanced type-level features are not supported
  • Unknown type identifiers are silently treated as string (graceful degradation)

Generation Limitations (Python Library)

  • Requires a HuggingFace causal LM — cannot use API-based models (OpenAI, Anthropic) directly through the library
  • Optional fields have a fixed 30% skip rate (not model-driven)
  • Nullable fields have a fixed 10% null rate (not model-driven)
  • Union types (non-enum) always pick the first non-null variant
  • Record<K, V> always generates exactly 3 key-value pairs
  • Array length is capped at max_array_length (default 10)
  • String length is capped at max_string_token_length tokens (default 10)

Web UI Limitations

  • Generation history is in-memory only — it resets when the server restarts
  • The Mistral API is the only supported backend for the web UI (the Python library supports any HuggingFace model)
  • No syntax highlighting in the TypeScript editor (it's a plain contenteditable div, not Monaco/CodeMirror)
  • No undo/redo in the editor

Models Tested

Python Library (Local)

Any HuggingFace AutoModelForCausalLM works. Tested with:

  • microsoft/phi-2 (2.7B)
  • mistralai/Mistral-7B-v0.1
  • meta-llama/Llama-2-7b-hf

Web UI (API)

  • mistral-small-latest (default)
  • Any Mistral model available via their chat completions API

Contributing

  1. Fork the repo
  2. Create a feature branch (git checkout -b feature/my-feature)
  3. Run tests (pytest tests/ -v)
  4. Submit a PR

License

MIT License — see LICENSE for details.

Copyright (c) 2026 Gustavo Flores (shellshock9001)


Built by shellshock9001
Inspired by jsonformer by Ramp

About

Constrained LLM output generation using TypeScript interfaces as the contract. The LLM never generates structure — only fills in values.

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors