Agent-X

    _                    _            __  __
   / \   __ _  ___ _ __ | |_         \ \/ /
  / _ \ / _` |/ _ \ '_ \| __|  _____  \  /
 / ___ \ (_| |  __/ | | | |_  |_____| /  \
/_/   \_\__, |\___|_| |_|\__|        /_/\_\
        |___/

A multi-capability AI agent system

Features: Vision, research, reasoning, computer use, and code execution

Navigation: Quick Start • Features • Documentation • Contributing

🌟 Features

🎨 Modern Web UI - React 19 chat interface with real-time streaming
🌊 Real-Time Streaming - Token-by-token, tool execution, and status updates via WebSocket/SSE
🤖 Multi-Modal AI - Vision, research, reasoning, and code execution
🔄 Universal Provider Support - Anthropic, OpenAI, Google, Ollama, vLLM, HuggingFace + more
🐳 Docker First - Complete containerized deployment with hot-reload
⚡ Fast & Scalable - Built with FastAPI, async patterns, and streaming
🛡️ Production Ready - Health checks, monitoring, logging, and rate limiting
🔧 Extensible - Plugin architecture for capabilities and providers

🚀 Quick Start

Web Interface (Recommended)

Get started with the web UI in 3 commands:

# 1. Setup environment
./agentx.sh setup

# 2. Start all services
./agentx.sh docker:up

# 3. Open your browser
open http://localhost:8080

Access Points:

🌐 Web UI: http://localhost:8080
🔌 API: http://localhost:8000
📚 API Docs: http://localhost:8000/docs

Command Line

# Interactive mode
./agentx.sh run

# Single command
./agentx.sh run "analyze this image: photo.png"

# Start API only
./agentx.sh api:start

Using Docker Compose Directly

# Start everything
docker-compose up -d

# Start with dev mode (hot-reload)
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# View logs
docker-compose logs -f

# Stop services
docker-compose down

📦 What's Included

Services

Service	Port	Description
Web Client	8080*	React 19 + TypeScript UI with real-time streaming
API Server	8000*	FastAPI backend with WebSocket/SSE support
PostgreSQL	5432*	Primary database for persistence
Redis	6379*	Caching, sessions, and rate limiting
Nginx	80/443	Reverse proxy (optional, production)

* Ports auto-resolve if conflicts detected

Capabilities

🖼️ Vision - Image analysis and understanding
🔍 Research - Web search and information gathering
💻 Code Execution - Safe sandboxed execution
🖱️ Computer Use - Browser automation
🔧 Function Calling - Custom tool integration
🧠 Reasoning - Complex problem solving

Supported Providers & Models

Agent-X supports 7+ LLM providers with universal streaming:

Provider	Models	Streaming	Vision	Tools
Anthropic	Claude 4 (Opus/Sonnet/Haiku), Claude 3.5	✅ Native	✅	✅
OpenAI	GPT-4, GPT-4 Turbo, GPT-3.5, O1	✅ Native	✅	✅
Google	Gemini Pro, Ultra, Flash, 1.5	✅ Native	✅	✅
Ollama	Llama 2/3, Mistral, Mixtral, CodeLlama, Phi	✅ Native	✅	✅
vLLM	Self-hosted models	✅ Native	⚠️	⚠️
HuggingFace	Any HF model	✅ Fallback	⚠️	⚠️
Local	Legacy local models	✅ Fallback	❌	❌

All providers support streaming through intelligent fallback mechanisms!

🛠️ Master Control Script

The agentx.sh script is your one-stop interface for everything:

./agentx.sh help  # Show all commands

Key Commands

# Setup & Environment
./agentx.sh setup              # Set up development environment
./agentx.sh env                # Show environment info

# Running Services
./agentx.sh run                # Run agent interactively
./agentx.sh api:start          # Start API server
./agentx.sh client:start       # Start web client
./agentx.sh docker:up          # Start all services

# Development
./agentx.sh client:dev         # Client with hot-reload
./agentx.sh test               # Run tests
./agentx.sh format             # Format code
./agentx.sh lint               # Lint code

# Docker Management
./agentx.sh docker:build       # Build all images
./agentx.sh docker:logs        # View logs
./agentx.sh docker:down        # Stop services
./agentx.sh docker:clean       # Clean up

# Database
./agentx.sh db:migrate         # Run migrations
./agentx.sh db:shell           # Open DB shell

See QUICKSTART.md for detailed guide.

🚦 Intelligent Port Resolution

Agent-X includes an automatic port conflict resolver that ensures smooth startup on any system:

# Automatically finds available ports when starting
./agentx.sh docker:up

How It Works

The port resolver runs automatically before Docker operations:

🔍 Detects Conflicts - Scans default ports (5432, 6379, 8000, 8080)
🔄 Finds Alternatives - If a port is in use, finds the next available one
⚡ Updates Config - Automatically updates docker-compose.override.yml
💾 Caches Results - Remembers port allocations for fast subsequent starts
✨ Transparent - All services communicate correctly with remapped ports

Features

⚡ Fast caching - Reuses ports from previous sessions
🛡️ Safe cleanup - Stops old Agent-X containers if needed
🔧 Auto-configuration - Updates .env and Docker config automatically
🎯 Smart fallback - Works even if detection tools aren't available

Manual Port Resolution

# Check port availability
./agentx.sh check-ports

# Force re-resolve ports
rm .ports && ./agentx.sh docker:up

Access URLs adapt automatically:

If 8080 is busy → Client might start on 8081
If 8000 is busy → API might start on 8001
Check .ports file for your actual URLs!

🏗️ Architecture

┌──────────────┐
│   Browser    │
└──────┬───────┘
       │ :8080 (auto-resolved)
  ┌────▼─────────┐
  │ agent-client │ (React 19 + TypeScript)
  │              │ WebSocket/SSE streaming
  └────┬─────────┘
       │ REST/WebSocket
  ┌────▼─────────┐
  │  agent-api   │ (FastAPI + Async)
  │              │ Universal provider support
  └────┬─────────┘
       │
  ┌────┴────┬────────┬─────────┬──────────┐
  │         │        │         │          │
┌─▼──┐  ┌──▼──┐  ┌──▼───┐  ┌──▼───┐  ┌──▼────┐
│Core│  │ DB  │  │Redis │  │Cache │  │Vector │
└────┘  └─────┘  └──────┘  └──────┘  └───────┘

Core Components

Orchestrator - ReAct pattern coordinator (Reason → Act → Observe)
LLM Abstraction - Model-agnostic provider layer
State Manager - Conversation & context tracking
Decision Engine - Request routing & capability selection
Capabilities - Specialized, pluggable modules

📚 Documentation

📖 QUICKSTART.md - Quick start guide
🐳 DOCKER.md - Docker documentation
💻 client/README.md - Web client docs
🔧 CLAUDE.md - Development with Claude Code
📁 docs/ - Full documentation

🧪 Development

Prerequisites

Python 3.11+
Docker & Docker Compose
Conda or Poetry (for local Python development)
Node.js 18+ & npm (for client development)

Local Setup

# Clone repository
git clone <repository-url>
cd agent-x

# Setup environment
./agentx.sh setup

# Configure API keys
cp .env.example .env
# Edit .env with your keys

# Start services
./agentx.sh docker:up

Development Mode

# Start everything with hot-reload
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up

# Or start individual services
./agentx.sh api:dev      # API with auto-reload
./agentx.sh client:dev   # Client with hot-reload

Running Tests

# All tests
./agentx.sh test

# By type
./agentx.sh test:unit
./agentx.sh test:integration
./agentx.sh test:e2e

# With coverage
./agentx.sh test:cov

# Specific file
./agentx.sh test tests/unit/test_orchestrator.py

Code Quality

# Format code
./agentx.sh format

# Lint
./agentx.sh lint

# Type check
./agentx.sh type

# Run all checks
./agentx.sh check

🔧 Configuration

Environment Variables

# LLM Provider API Keys (at least one required)
ANTHROPIC_API_KEY=your_key_here
OPENAI_API_KEY=your_key_here
GOOGLE_API_KEY=your_key_here
HUGGINGFACE_API_KEY=your_key_here

# Default Provider & Model
DEFAULT_PROVIDER=anthropic
DEFAULT_MODEL=claude-sonnet-4

# Agent Behavior
MAX_ITERATIONS=10
TIMEOUT=300
ENABLE_MEMORY=true
MAX_HISTORY_LENGTH=100

# Streaming Configuration
ENABLE_STREAMING=false  # Master switch for streaming features
# When enabled, provides:
# - Token-by-token streaming
# - Tool execution events
# - Status updates
# - Optional thinking process

# Service URLs (auto-resolved if ports conflict)
API_URL=http://localhost:8000
CLIENT_URL=http://localhost:8080

# Database (port auto-resolves)
DATABASE_URL=postgresql://agentx:password@localhost:5432/agentx

# Redis (port auto-resolves)
REDIS_URL=redis://localhost:6379/0

# Rate Limiting (disabled by default for development)
ENABLE_RATE_LIMITING=false
RATE_LIMIT_REQUESTS_PER_MINUTE=1000
RATE_LIMIT_TOKENS_PER_MINUTE=2000000

# Cost Controls
COST_TRACKING_ENABLED=true
MAX_COST_PER_REQUEST=1.0
MAX_COST_PER_DAY=100.0

Docker Configuration

Edit docker-compose.yml or docker-compose.dev.yml:

services:
  agent-api:
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - WORKERS=4
      - DEBUG=true

📊 Project Structure

agent-x/
├── agentx.sh              # Master control script
├── scripts/               # Utility scripts
│   ├── port-resolver.sh  # Auto port conflict resolution
│   └── resolve-ports.sh  # Manual port resolver
├── client/                # Web UI (React 19 + TypeScript + Vite)
│   ├── src/
│   │   ├── components/   # React UI components
│   │   ├── stores/       # Zustand state management
│   │   ├── api/          # API client & WebSocket
│   │   └── hooks/        # Custom React hooks
│   ├── vite.config.ts    # Vite config with proxy
│   └── Dockerfile        # Client container
├── core/                  # Core orchestrator & engine
│   ├── orchestrator.py   # Main ReAct loop with streaming
│   ├── agent-x.py        # High-level AgentX API
│   ├── state_manager.py  # State & conversation management
│   ├── decision_engine.py # Request routing
│   └── scheduler.py      # Task scheduling
├── llm/                   # LLM provider abstraction
│   ├── base_provider.py  # Base provider with streaming
│   ├── anthropic_provider.py  # Claude provider
│   ├── openai_provider.py     # OpenAI provider
│   ├── google_provider.py     # Gemini provider
│   ├── ollama_provider.py     # Ollama provider
│   ├── vllm_provider.py       # vLLM provider
│   ├── hf_provider.py         # HuggingFace provider
│   ├── factory.py        # Provider factory
│   └── model_manager.py  # Model management
├── capabilities/          # Capability modules
│   ├── vision/           # Vision capability
│   ├── research/         # Research capability
│   ├── code_execution/   # Code execution
│   ├── manager.py        # Capability manager
│   └── base_capability.py # Base interface
├── api/                   # FastAPI server
│   ├── app.py            # Main API app
│   ├── routes.py         # REST API routes
│   ├── streaming.py      # SSE streaming endpoints
│   ├── websocket.py      # WebSocket handler
│   ├── schemas.py        # Pydantic models
│   └── middleware.py     # API middleware
├── config/                # Configuration
│   ├── settings.py       # Settings with streaming config
│   └── llm.py            # LLM configurations
├── safety/                # Safety & rate limiting
│   ├── rate_limiter.py   # Rate limiting
│   └── checker.py        # Safety checks
├── storage/               # Persistence layer
│   ├── database.py       # Database management
│   └── cache.py          # Redis caching
├── memory/                # Memory systems
│   ├── conversation.py   # Short-term memory
│   └── long_term.py      # Vector store
├── evaluation/            # Evaluation & benchmarking
│   ├── evaluator.py      # Task evaluator
│   └── llm_judge.py      # LLM-based evaluation
├── tests/                 # Test suite
│   ├── unit/             # Unit tests
│   ├── integration/      # Integration tests
│   └── e2e/              # End-to-end tests
├── docs/                  # Documentation
├── docker-compose.yml     # Production compose
├── docker-compose.dev.yml # Development compose
└── Dockerfile             # Main Python container

🌐 API Usage

REST API

# Send message
curl -X POST http://localhost:8000/api/v1/agent/run \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Analyze this image",
    "model": "claude-sonnet-4",
    "context": {}
  }'

Server-Sent Events (SSE) Streaming

# Stream response with SSE
curl -N -X POST http://localhost:8000/api/v1/stream/agent \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Tell me a story",
    "stream_tokens": true,
    "stream_thinking": false
  }'

WebSocket Streaming

// Connect to WebSocket endpoint
const ws = new WebSocket('ws://localhost:8000/api/v1/ws/agent');

// Send message
ws.send(JSON.stringify({
  message: "Hello, Agent-X!",
  session_id: "my-session",
  model: "claude-sonnet-4"
}));

// Receive streaming updates
ws.onmessage = (event) => {
  const data = JSON.parse(event.data);

  switch(data.type) {
    case 'token':
      process.stdout.write(data.content.token);
      break;
    case 'tool_call':
      console.log(`\nTool: ${data.content.tool_name}`);
      break;
    case 'status':
      console.log(`Status: ${data.content.status}`);
      break;
    case 'complete':
      console.log('\nDone!');
      break;
  }
};

Python Client

from core.agent_x import AgentX

# Initialize agent
agent = AgentX(streaming=True)

# Streaming with callbacks
def on_token(token):
    print(token, end="", flush=True)

def on_tool(tool_name, params, result):
    if result:
        print(f"\n✓ Tool {tool_name} completed")

# Stream with callbacks
for chunk in agent.stream(
    "Explain quantum computing",
    on_token=on_token,
    on_tool=on_tool
):
    if chunk["done"]:
        print(f"\nMetadata: {chunk['metadata']}")

Streaming Features

Agent-X provides real-time streaming for all providers:

Token Streaming - Token-by-token output as it's generated
Tool Execution - Real-time notifications when tools are called
Status Updates - Progress indicators during long operations
Thinking Process - Optional agent reasoning visibility
Universal Support - Works with ALL 7+ providers (native or fallback)

Streaming Protocols:

🔌 WebSocket - Full bidirectional communication
📡 SSE - Server-Sent Events for one-way streaming
🔄 Fallback - Automatic provider fallback for universal compatibility

🚀 Deployment

Production Build

# Build production images
./agentx.sh docker:build

# Start with production profile
docker-compose --profile production up -d

# Enable nginx reverse proxy
docker-compose --profile production up nginx

Environment Variables for Production

# Required
ANTHROPIC_API_KEY=sk-...
DATABASE_URL=postgresql://...
REDIS_URL=redis://...

# Recommended
API_WORKERS=4
LOG_LEVEL=INFO
CORS_ORIGINS=https://yourdomain.com
SSL_CERT_PATH=/etc/ssl/cert.pem
SSL_KEY_PATH=/etc/ssl/key.pem

🐛 Troubleshooting

Services won't start

# Check Docker
docker info

# Check for port conflicts
lsof -i :8000  # API
lsof -i :8080  # Client

# Clean and rebuild
./agentx.sh docker:clean
./agentx.sh docker:build
./agentx.sh docker:up

Client can't connect to API

# Check API is running
curl http://localhost:8000/health

# Check client environment
docker-compose exec agent-client env | grep API_URL

# Check logs
./agentx.sh api:logs
./agentx.sh client:logs

Database issues

# Reset database
./agentx.sh db:reset

# Run migrations
./agentx.sh db:migrate

# Check connection
docker-compose exec postgres psql -U agentx -d agentx

See QUICKSTART.md for more solutions.

🤝 Contributing

We welcome contributions! Please follow these steps:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes
Test your changes (./agentx.sh check && ./agentx.sh test)
Commit (git commit -m 'Add amazing feature')
Push (git push origin feature/amazing-feature)
Open a Pull Request

Development Guidelines

Follow the code style (use ./agentx.sh format)
Add tests for new features
Update documentation
Ensure all checks pass

See CLAUDE.md for detailed development guidelines.

📄 License

[Your License Here]

💬 Support

Issues: GitHub Issues
Documentation: Run ./agentx.sh help
Quick Start: QUICKSTART.md
Development: CLAUDE.md

🙏 Acknowledgments

Built with Anthropic Claude
Powered by FastAPI
UI built with React and Vite
Streaming support inspired by modern real-time architectures

⭐ Key Features Summary

🚀 Production-Ready Architecture

Universal Provider Support - 7+ LLM providers with automatic fallback
Real-Time Streaming - WebSocket and SSE with token-by-token output
Intelligent Port Resolution - Zero-config startup on any system
Docker-First Design - Complete containerization with hot-reload
Modern Web UI - React 19 with real-time streaming interface

🛡️ Enterprise Features

Rate Limiting - Token and request-based rate limiting with Redis
Cost Tracking - Per-request, daily, and monthly cost controls
Health Checks - Comprehensive monitoring and diagnostics
Database Persistence - PostgreSQL with automatic migrations
Vector Store - Long-term memory with Chroma
Caching - Redis-based caching for performance

🔧 Developer Experience

Master Control Script (agentx.sh) - One command for everything
Hot-Reload - Auto-reload for API and client during development
Comprehensive Tests - Unit, integration, and E2E test suites
Type Safety - Full type hints and mypy checking
Code Quality - Black, isort, ruff, and pre-commit hooks

🌐 API & Integration

REST API - Full-featured HTTP API
WebSocket - Real-time bidirectional communication
SSE Streaming - Server-Sent Events for one-way streaming
Python SDK - High-level AgentX class with streaming support
OpenAPI Docs - Auto-generated API documentation

📈 Version

Current version: 0.1.0

./agentx.sh version

🗺️ Roadmap

Enhanced multi-modal capabilities (audio, video)
Advanced reasoning modes (chain-of-thought, tree-of-thought)
Distributed execution for parallel tool use
Plugin marketplace for custom capabilities
Cloud deployment templates (AWS, GCP, Azure)
Enterprise SSO and RBAC

⬆ back to top

Made with ❤️ by the Agent-X Team

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
api		api
capabilities		capabilities
client		client
config		config
core		core
docker		docker
docs		docs
evaluation		evaluation
llm		llm
memory		memory
observability		observability
safety		safety
scripts		scripts
setup		setup
storage		storage
tests		tests
tools		tools
utils		utils
.dockerignore		.dockerignore
.env.docker-compose		.env.docker-compose
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
DOCKER.md		DOCKER.md
Dockerfile		Dockerfile
HowTo.md		HowTo.md
QUICKSTART.md		QUICKSTART.md
README.md		README.md
__init__.py		__init__.py
agentx.sh		agentx.sh
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.yml		docker-compose.yml
docker-entrypoint.sh		docker-entrypoint.sh
environment.yml		environment.yml
main.py		main.py
package-lock.json		package-lock.json
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

Agent-X

A multi-capability AI agent system

🌟 Features

🚀 Quick Start

Web Interface (Recommended)

Command Line

Using Docker Compose Directly

📦 What's Included

Services

Capabilities

Supported Providers & Models

🛠️ Master Control Script

Key Commands

🚦 Intelligent Port Resolution

How It Works

Features

Manual Port Resolution

🏗️ Architecture

Core Components

📚 Documentation

🧪 Development

Prerequisites

Local Setup

Development Mode

Running Tests

Code Quality

🔧 Configuration

Environment Variables

Docker Configuration

📊 Project Structure

🌐 API Usage

REST API

Server-Sent Events (SSE) Streaming

WebSocket Streaming

Python Client

Streaming Features

🚀 Deployment

Production Build

Environment Variables for Production

🐛 Troubleshooting

Services won't start

Client can't connect to API

Database issues

🤝 Contributing

Development Guidelines

📄 License

💬 Support

🙏 Acknowledgments

⭐ Key Features Summary

🚀 Production-Ready Architecture

🛡️ Enterprise Features

🔧 Developer Experience

🌐 API & Integration

📈 Version

🗺️ Roadmap

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages