Ax is a TypeScript framework for building reliable AI applications with Large Language Models (LLMs). Inspired by Stanford's DSPy (Declarative Self-improving Language Programs), Ax separates what you want (typed input/output contracts called signatures) from how to achieve it (automatic prompt generation and optimization). Instead of manually engineering prompts, developers define signatures like "question:string -> answer:string" using the AxSignature class src/dsp/sig.ts300-364 and the framework handles prompt construction via AxPromptTemplate src/dsp/prompt.ts26-207 response parsing through extractValues() src/dsp/generate.ts1005-1193 and automatic performance optimization.
Core Philosophy: Treat LLM interactions as programmable modules with strong contracts, not fragile text prompts. The framework provides compile-time type safety, automatic validation, and self-improvement capabilities that eliminate the need for manual prompt iteration.
Sources: README.md1-11 src/dsp/sig.ts300-364 src/dsp/prompt.ts26-207 src/dsp/generate.ts1005-1193
Traditional LLM development requires manual prompt engineering, fragile text parsing, and provider-specific code. Ax eliminates these pain points through:
AxSignature system src/dsp/sig.ts300-364 - specify inputs and outputs, not promptsAxPromptTemplate src/dsp/prompt.ts26-207 response parsing through extractValues() src/dsp/generate.ts1005-1193 and validation src/dsp/sig.ts828-891AxAIService interface src/ai/types.ts works across 15+ LLM providers with automatic feature detectionAxMiPRO src/dsp/optimizers/miproV2.ts AxGEPA src/dsp/optimizers/gepa.ts and AxACE src/dsp/optimizers/ace.tsSources: src/dsp/sig.ts300-891 src/dsp/prompt.ts26-207 src/dsp/generate.ts1005-1193 src/ai/types.ts src/dsp/optimizers/
From Signature to Execution
The framework provides five core abstractions:
| Abstraction | Class | Purpose | Factory Function |
|---|---|---|---|
| Signature | AxSignature | Type-safe input/output contract | ax(), f() src/dsp/template.ts18-62 src/dsp/sig.ts1097-1157 |
| Program | AxGen | Executable unit implementing signature | ax() returns AxGen src/dsp/generate.ts77-240 |
| Agent | AxAgent | Program with function calling (ReAct) | agent() src/prompts/agent.ts682-731 |
| Workflow | AxFlow | Multi-step orchestration with state | flow() src/flow/flow.ts40-580 |
| Service | AxAI | Unified interface to LLM providers | ai() src/ai/wrap.ts300-346 |
Sources: README.md1-236 src/dsp/sig.ts300-364 src/dsp/sig.ts1097-1157 src/dsp/generate.ts77-240 src/ai/types.ts src/dsp/template.ts18-62 src/prompts/agent.ts682-731 src/flow/flow.ts40-580 src/ai/wrap.ts300-346
Ax is organized into seven conceptual layers, from user-facing abstractions down to infrastructure:
Framework Architecture Layers
Layer Responsibilities:
| Layer | Components | Responsibilities |
|---|---|---|
| User Applications | Application code, examples | Entry point for building AI applications |
| High-Level Abstractions | AxAgent, AxFlow, AxProgram, AxGen | Programmable units with typed interfaces |
| DSP Layer | AxSignature, AxPromptTemplate, extractors, assertions | Signature-driven prompt engineering and validation |
| AI Service Abstraction | AxAI, AxBaseAI, routers | Unified interface to LLM providers |
| Provider Implementations | 15+ provider implementations | Provider-specific API transformations |
| Optimization Layer | AxMiPRO, BootstrapFewShot, AxACE, AxGEPA | Automatic program improvement |
| Infrastructure | API calls, memory, response processing, errors | Core utilities and state management |
| Supporting Systems | Telemetry, logging, caching | Observability and performance optimization |
Automatic Operations: The framework handles these operations without developer intervention:
| Operation | Implementation | Purpose |
|---|---|---|
| Prompt Generation | AxPromptTemplate.render() src/dsp/prompt.ts109-207 | Converts signatures to provider-specific prompts |
| Response Parsing | extractValues() src/dsp/generate.ts1005-1193 | Extracts typed fields from LLM responses |
| Type Validation | AxSignature.validate() src/dsp/sig.ts828-891 | Ensures inputs/outputs match signature types |
| Streaming | streamingExtractValues() src/dsp/generate.ts699-926 | Incremental parsing during streaming responses |
| Error Correction | assertAssertions() src/dsp/generate.ts928-1003 | Automatic retry with fixing instructions |
| Optimization | AxBaseOptimizer.compile() src/dsp/optimizer.ts651-723 | Automatic program tuning with examples |
Sources: src/prompts/agent.ts178-513 src/flow/flow.ts40-580 src/dsp/program.ts18-227 src/dsp/generate.ts77-1193 src/dsp/sig.ts300-891 src/dsp/prompt.ts26-207 src/dsp/asserts.ts src/ai/wrap.ts69-181 src/ai/base.ts148-421 src/ai/router.ts src/ai/balance.ts src/ai/openai/api.ts src/ai/anthropic/api.ts src/ai/google-gemini/api.ts src/dsp/optimizers/miproV2.ts src/dsp/optimizers/bootstrapFewshot.ts src/dsp/optimizers/ace.ts src/dsp/optimizers/gepa.ts src/util/apicall.ts src/mem/memory.ts src/dsp/processResponse.ts src/dsp/loggers.ts
The monorepo contains six npm packages:
Monorepo Packages
| Package | Version | Location | Dependencies |
|---|---|---|---|
@ax-llm/ax | 16.0.9 | src/ax/ | @opentelemetry/api@^1.9.0, dayjs@^1.11.13 |
@ax-llm/ax-tools | 16.0.9 | src/tools/ | @ax-llm/ax |
@ax-llm/ax-ai-sdk-provider | 16.0.9 | src/aisdk/ | @ax-llm/ax, ai@^5.0.100, @ai-sdk/provider-utils@^3.0.10 |
@ax-llm/ax-ai-aws-bedrock | 16.0.9 | src/aws-bedrock/ | @ax-llm/ax, @aws-sdk/client-bedrock-runtime@^3.709.0 |
@ax-llm/ax-examples | 11.0.61 | src/examples/ | @ax-llm/ax, @ax-llm/ax-tools, OpenTelemetry SDK |
@ax-llm/ax-docs | - | src/docs/ | Astro, React, Tailwind |
The workspace uses npm workspaces package.json71-74 with unified tooling: TypeScript 5.8+ package.json61 Vitest 3.2+ package.json62 for testing, Biome 2.1+ package.json40 for linting/formatting, tsup 8.5+ package.json56 for bundling. Minimum Node.js version is 20 package.json37-38
Sources: package.json1-78 src/ax/package.json1-70 src/aisdk/package.json1-47 src/tools/package.json1-46 src/examples/package.json1-41 src/aws-bedrock/package.json1-54
When you call program.forward(ai, input), the request flows through six stages:
Core Execution Flow - From User Code to LLM Response
Pipeline Stages:
AxSignature.validate() checks types and required fieldsAxPromptTemplate.render() formats signature, examples, and demos with cache flagsAxMemory.addRequest() stores user message in conversation historyAxBaseAI.chat() applies rate limiting, tracing, and delegates to providercreateChatReq() and createChatResp() methodsstreamingExtractValues() src/dsp/generate.ts699-926 with incremental parsingextractValues() src/dsp/generate.ts1005-1193 with complete parsingassertAssertions() runs checks, retries on failureAxFunctionProcessor executes tools and loops if neededAutomatic Retry Logic: If assertions fail at stage 7, the framework automatically:
maxRetries times, default 3)Sources: src/dsp/generate.ts196-1193 src/dsp/sig.ts828-891 src/dsp/prompt.ts26-207 src/mem/memory.ts63-270 src/ai/base.ts220-421 src/ai/openai/api.ts192-511 src/dsp/functions.ts205-457 src/ai/types.ts
Ax abstracts 15+ LLM providers behind a unified AxAIService interface src/ai/types.ts:
Multi-Provider Architecture
Provider Implementation Pattern: Each provider extends AxBaseAI and implements:
createChatReq() - Transform AxChatRequest to provider formatcreateChatResp() - Parse provider response to AxChatResponsecreateChatStreamResp() - Handle streaming responsesAdvanced Routing Options:
Sources: src/ai/wrap.ts69-181 src/ai/base.ts148-421 src/ai/types.ts src/ai/balance.ts40-299 src/ai/multiservice.ts src/ai/openai/api.ts src/ai/anthropic/api.ts src/ai/google-gemini/api.ts
Here's what happens when you run this code:
Detailed Execution Sequence with Code Entities
Key Operations:
AxAIOpenAI providerAxSignature, creates AxGenreview:string fieldYou are a classifier. Extract the following fields:
- sentiment: class (one of: positive, negative, neutral)
Input:
review: "Great product!"
Sources: src/ai/wrap.ts300-346 src/dsp/template.ts18-62 src/dsp/parser.ts58-139 src/dsp/generate.ts77-1193 src/dsp/sig.ts828-891 src/dsp/prompt.ts109-207 src/ai/base.ts220-421 src/ai/openai/api.ts
The framework provides four primary entry points for creating AI components:
Factory Function Overview
| Factory | Returns | Purpose | Code Location |
|---|---|---|---|
ai() | AxAI | Create AI service instances | src/ai/wrap.ts300-346 |
ax() | AxGen | Parse signatures into programs | src/dsp/template.ts549 |
agent() | AxAgent | Create multi-agent systems | src/prompts/agent.ts682-731 |
flow() | AxFlow | Build workflow orchestrations | src/flow/flow.ts582 |
Sources: src/ai/wrap.ts69-346 src/dsp/template.ts18-62 src/prompts/agent.ts682-731 src/flow/flow.ts40-582
Ax provides compile-time and runtime type safety through its signature system:
Signature Definition to Runtime Validation
The type system is implemented across multiple layers:
Supported Field Types src/dsp/sig.ts17-31:
| Type Category | Types | Parsed Representation |
|---|---|---|
| Primitives | string, number, boolean | Direct JSON values |
| Complex | json, class, array | Nested objects, enums, lists |
| Media | image, audio, file | Base64 data URIs or URLs |
| Temporal | datetime | ISO 8601 strings |
Field Constraints and Modifiers:
field?:type parsed by SignatureParser.parseField() src/dsp/parser.ts141-236field[]:type creates AxFieldArrayType src/dsp/sig.ts32-39 with element type trackingfield:class "val1, val2, val3" with enumerated values stored in AxSignature._classes src/dsp/sig.ts237-265field:type "description text" or via fluent API f().input("field", f.string("desc"))Three-Stage Validation Pipeline:
SignatureParser.validateFieldName() src/dsp/parser.ts309-344 enforces naming conventions (alphanumeric + underscore, starts with letter)AxSignature.validate() src/dsp/sig.ts828-891 checks required fields and types before executionvalidateAndParseFieldValue() src/dsp/util.ts13-190 parses LLM output and enforces type complianceTypeScript Type Inference: Generic parameters AxSignature<IN, OUT> src/dsp/sig.ts300-364 enable full autocomplete for forward(ai, input: IN): Promise<OUT> via InferFieldValueType src/dsp/sig.ts41-96 utility type.
Sources: src/dsp/sig.ts17-891 src/dsp/parser.ts58-344 src/dsp/util.ts13-190
The framework provides a unified interface for 15+ LLM providers through a three-layer architecture:
Provider Abstraction Layers
Each provider extending AxBaseAI src/ai/base.ts148-219 must implement three core methods:
Required Methods:
createChatReq(req: AxChatRequest): ProviderRequest - Transform unified request to provider-specific formatcreateChatResp(resp: ProviderResponse): AxChatResponse - Parse provider response to unified formatcreateChatStreamResp(stream: SSEResponse): AsyncGenerator<AxChatResponse> - Handle streaming with SSE parsing via streamSSE() src/util/apicall.ts190-315Provider-Specific Capabilities:
| Feature | Implementation | Example Providers |
|---|---|---|
| Thinking tokens | thinking config property | AxAIGoogleGemini src/ai/google-gemini/api.ts270-320 AxAIAnthropic src/ai/anthropic/api.ts180-273 |
| Schema cleanup | Remove unsupported JSON Schema properties | cleanSchema() in AxAIGoogleGemini src/ai/google-gemini/api.ts468-492 |
| Multi-modal | Content part transformation | processContent() in AxAIOpenAI src/ai/openai/api.ts315-425 |
| Function calling | Native tool support vs. prompt-based | AxAICohere uses tool_results src/ai/cohere/api.ts110-215 |
| Embeddings | embed() method implementation | AxAIOpenAI.embed() src/ai/openai/api.ts513-594 |
Schema Constraints Example: Google Gemini does not support additionalProperties in JSON Schema, requiring schema transformation in createChatReq() src/ai/google-gemini/api.ts468-492
Sources: src/ai/base.ts148-421 src/ai/google-gemini/api.ts136-492 src/ai/anthropic/api.ts180-273 src/ai/openai/api.ts315-594 src/ai/cohere/api.ts110-215 src/util/apicall.ts190-315
Ax provides three optimization algorithms for automatically improving AI program performance:
Optimizer Architecture
| Optimizer | Algorithm | Implementation | Best For |
|---|---|---|---|
| AxMiPRO | Multi-objective Bayesian optimization (MIPROv2) | src/dsp/optimizers/miproV2.ts40-867 | Multi-objective optimization with instruction and example tuning, no external dependencies |
| AxGEPA | Gradient Evolution + Pareto Analysis | src/dsp/optimizers/gepa.ts | Multi-objective optimization with Pareto-optimal solution discovery |
| AxGEPAFlow | Multi-module GEPA | src/dsp/optimizers/gepaFlow.ts | Complex workflows with multiple interconnected AxGen steps |
| AxACE | Agentic Context Engineering | src/dsp/optimizers/ace.ts1-848 | Iterative playbook refinement via generator→reflector→curator loops |
| AxBootstrapFewShot | Trace-based demo generation | src/dsp/optimizers/bootstrapFewshot.ts15-180 | Few-shot learning from successful execution traces |
| AxLearn | Unified optimization API | src/dsp/learn.ts9-211 | Simplified interface replacing deprecated AxLearnAgent and AxTuner (v15.0.25+) |
AxLearn Unified Interface: Introduced in version 15.0.25 CHANGELOG.md129-136 AxLearn provides a MIPROv2-like API that consolidates optimization capabilities previously split across multiple classes. It supports instruction optimization, demo bootstrapping, and parameter tuning through a single optimize() method src/dsp/learn.ts29-199
Sources: src/dsp/optimizers/miproV2.ts40-867 src/dsp/optimizers/gepa.ts src/dsp/optimizers/gepaFlow.ts src/dsp/optimizers/ace.ts1-848 src/dsp/optimizers/bootstrapFewshot.ts15-180 src/dsp/learn.ts9-211 CHANGELOG.md129-136
Common Optimizer Interface: All optimizers extend AxBaseOptimizer src/dsp/optimizer.ts400-650 providing:
compile(options: AxCompileOptions): Promise<AxOptimizerResult> src/dsp/optimizer.ts651-723 - Main optimization loop with metric evaluationAxOptimizedProgram result src/dsp/optimizer.ts302-363 - Serializable configuration with toJSON() and applyOptimization()AxOptimizationCheckpoint src/dsp/common_types.ts185-203 with save/load callbacksAxDefaultCostTracker src/dsp/optimizer.ts42-58 to enforce budget limitsAxOptimizerLoggerFunction src/dsp/optimizerTypes.ts5-65AxLearn Unified API: The AxLearn class src/dsp/learn.ts provides a simplified interface that replaced AxLearnAgent and AxTuner in version 15.0.25, offering a MIPROv2-like API for program optimization.
External Dependencies: All optimizers are pure TypeScript implementations with no external dependencies. Early versions of MiPRO required a Python service for Bayesian optimization, but this was eliminated in later releases.
Sources: src/dsp/optimizer.ts42-900 src/dsp/optimizers/miproV2.ts40-867 src/dsp/optimizers/gepa.ts src/dsp/optimizers/gepaFlow.ts src/dsp/optimizers/ace.ts1-848 src/dsp/optimizers/bootstrapFewshot.ts15-180 src/dsp/learn.ts9-211 src/dsp/common_types.ts185-203 src/dsp/optimizerTypes.ts5-65
The framework implements multi-level error handling with automatic retry and correction mechanisms:
Error Handling & Resilience Architecture
Error Type Hierarchy [src/util/apicall.ts]:
| Error Type | Retry Behavior | Use Case |
|---|---|---|
AxAIServiceNetworkError | Automatic retry with exponential backoff | Network failures, DNS issues |
AxAIServiceTimeoutError | Automatic retry with increased timeout | Slow responses, timeouts |
AxAIServiceStatusError | Retry on 5xx, fail on 4xx | Server errors vs client errors |
AxTokenLimitError | No retry (context too large) | Token limit exceeded |
AxAIServiceAuthenticationError | No retry | Invalid API key |
AxAIRefusalError | No retry | Content policy violation |
AxAssertionError | Validation retry with correction | Output validation failures |
Three-Level Retry Strategy:
Infrastructure Retry [src/util/apicall.ts]: Automatic retry for network/5xx/timeout errors
Retry-After headers from rate limitingretryConfig on API optionsValidation Retry src/dsp/generate.ts928-1003: Automatic retry for assertion failures
maxRetries times)Multi-Step Retry src/dsp/generate.ts196-240: Loops for function calling
maxSteps limit (default: 10)Error Recovery Mechanisms:
Sources: src/util/apicall.ts src/dsp/generate.ts196-1003 src/ai/balance.ts src/dsp/asserts.ts
Context Caching System
Context caching reduces costs and latency by reusing expensive prompt prefixes (system prompts, examples, function definitions). Supported providers: Anthropic src/ai/anthropic/api.ts Google Gemini src/ai/google-gemini/api.ts
Cache Breakpoints:
system: Cache system prompt onlyafter-functions: Cache system prompt + function definitionsafter-examples: Cache system prompt + functions + few-shot examplesSources: src/ai/anthropic/api.ts src/ai/google-gemini/api.ts CHANGELOG.md39-73
Streaming Architecture
Streaming provides real-time response delivery with incremental value extraction. The framework maintains state across chunks and yields only new content deltas src/dsp/generate.ts699-926
Sources: src/dsp/generate.ts699-926 src/util/apicall.ts190-315 src/dsp/processResponse.ts
Memory System
AxMemory src/mem/memory.ts63-270 maintains conversational state across sessions with tag-based checkpointing for error recovery.
Sources: src/mem/memory.ts63-270 src/dsp/functions.ts205-457
AxFlow src/flow/flow.ts40-580 provides declarative workflow construction with automatic dependency analysis and parallel execution:
AxFlow Features:
AxFlowDependencyAnalyzer.analyze() src/flow/dependencyAnalyzer.ts17-129 builds dependency graphsAxFlowExecutionPlanner.plan() src/flow/executionPlanner.ts82-227 computes execution levels and parallelizationTState parameter provides compile-time safetybranch(), while(), parallel(), feedback() primitives src/flow/flow.ts166-473Sources: src/flow/flow.ts40-582 src/flow/dependencyAnalyzer.ts17-129 src/flow/executionPlanner.ts82-227 src/flow/subContext.ts12-124
agent() factory src/prompts/agent.ts682-731 creates hierarchical agents with function calling:
Agent Features:
AxFunctionProcessor src/dsp/functions.ts205-457 handles tool loopsprocessChildAgentFunction() src/prompts/agent.ts59-141addModelParameter() src/prompts/agent.ts545-590 enables model selectionnative, prompt, auto execution src/dsp/functions.ts205-457Sources: src/prompts/agent.ts59-731 src/dsp/functions.ts67-457 src/ai/base.ts196-218
Comprehensive telemetry through OpenTelemetry:
Telemetry System
Metrics Instruments:
| Instrument Category | Metrics | Implementation |
|---|---|---|
| Generation | Request count, latency (p50/p95/p99), retry count, assertion failures | AxGenMetricsInstruments src/dsp/metrics.ts48-76 |
| AI Service | Token usage (input/output), cost estimates, request/response sizes, error rates | AxAIMetricsInstruments src/ai/metrics.ts14-43 |
| Optimization | Compilation rounds, best score progression, convergence rate, evaluated configurations | AxOptimizerMetricsInstruments src/dsp/optimizer.ts60-91 |
Configuration via axGlobals: Set OpenTelemetry components globally via axGlobals src/dsp/globals.ts1-16:
| Component | Purpose | Key Methods | Location |
|---|---|---|---|
AxMemory | Multi-session conversation history | add(), addTag(), reset(), history() | src/mem/memory.ts63-270 |
AxDB | Vector database abstraction | upsert(), query() | src/db/wrap.ts11-77 |
AxDBManager | RAG with query rewriting and reranking | batchUpsert(), query(), search() | src/docs/manager.ts38-335 |
axGlobals | Global framework configuration | tracer, meter, logger, functionResultFormatter | src/dsp/globals.ts1-16 |
AxMemory Features:
sessionId maintains independent conversation historyaddTag(sessionId, tag) src/mem/memory.ts194-214 creates named checkpoints, resetToTag() reverts to saved stateupdateMessage() src/mem/memory.ts161-171 allows incremental message modification during streaming responseshistory(sessionId, count) src/mem/memory.ts216-243 retrieves conversation context with optional message limitVector Database Implementations:
| Provider | Implementation | Key Methods |
|---|---|---|
| Pinecone | AxDBPinecone src/db/pinecone.ts | upsert(), query(), namespace-based partitioning |
| Weaviate | AxDBWeaviate src/db/weaviate.ts | upsert(), query(), schema management |
| Cloudflare | AxDBCloudflare src/db/cloudflare.ts | upsert(), query(), edge-deployed vectorization |
| In-Memory | AxDBMemory src/db/memory.ts | upsert(), query(), ephemeral storage for testing |
All implementations conform to AxDBService interface src/db/types.ts with standardized upsert() and query() signatures.
| Integration | Component | Purpose | Location |
|---|---|---|---|
| Vercel AI SDK | AxAIProvider | Bridge to Vercel AI SDK v5 | src/aisdk/provider.ts |
| Apache Tika | AxApacheTika | Document format conversion (PDF, DOCX, etc.) | src/docs/tika.ts14-100 |
| MCP Protocol | AxMCPClient | Model Context Protocol client for server tools | src/mcp/client.ts26-203 |
| MCP Transports | AxMCPHTTPSSETransport, AxMCPStreambleHTTPTransport | HTTP/SSE transport implementations | src/mcp/transports/ |
| Docker | AxDockerSession | Sandboxed code execution environment | src/funcs/docker.ts12-156 |
MCP Transport Options:
AxMCPHTTPSSETransport src/mcp/transports/sseTransport.ts for server-sent eventsAxMCPStreambleHTTPTransport src/mcp/transports/httpStreamTransport.ts for bidirectional streamingAxMCPOAuthOptions src/mcp/oauth/types.ts with PKCE for secure authenticationAxDockerSession Usage: Provides sandboxed Python/Node.js code execution with configurable resource limits src/funcs/docker.ts12-156 Example:
Sources: src/aisdk/provider.ts src/docs/tika.ts14-100 src/mcp/client.ts26-203 src/mcp/transports/sseTransport.ts src/mcp/transports/httpStreamTransport.ts src/mcp/oauth/types.ts src/funcs/docker.ts12-156 src/dsp/generate.ts196-240 src/ai/base.ts220-421 src/dsp/globals.ts1-16 src/mem/memory.ts63-270 src/db/wrap.ts11-77 src/db/types.ts src/docs/manager.ts38-335
The framework follows semantic versioning with synchronized package versions:
Build artifacts are generated using tsup and published to npm with provenance. The monorepo uses npm workspaces for managing inter-package dependencies.
Sources: package.json1-78 CHANGELOG.md1-204
Refresh this wiki
This wiki was recently refreshed. Please wait 3 days to refresh again.