VisionFlow: AI Agent Swarm Visualisation & Knowledge Graph Explorer

Experience the future of knowledge management: Real-time 3D visualisation of AI agent swarms working alongside your Logseq knowledge graph.

Inspired by the innovative work of Prof. Rob Aspin: https://github.com/trebornipsa

---

🚀 Quick Start

# Clone and deploy in minutes
git clone <repository-url>
cd ext

# Configure environment
cp .env.example .env

# Deploy with Docker
docker-compose up -d

# Access the application
open http://localhost:3001

📚 Full Quick Start Guide - Spawn your first AI swarm in 5 minutes!

---

📊 System Architecture Documentation

🔍 Complete System Diagrams - Comprehensive architecture diagrams covering all data flows from user input to GPU rendering, including:

• System overview and component architecture
• Client-server real-time communication flows
• Actor system message passing
• GPU compute pipeline details
• Authentication and authorization flows
• External services integration
• And much more...

---

Overview

VisionFlow bridges knowledge management and AI agent orchestration through:

• High-Performance Backend: Rust + Actix actor system
• 3D Visualisation: React + Three.js with WebGL/WebXR
• GPU Acceleration: CUDA physics for 200+ agents at 60 FPS
• Binary Protocol: 85% bandwidth reduction vs JSON
• Distributed Consensus: Raft, Byzantine, CRDT protocols
• AI Integration: Claude Flow MCP with 70+ orchestration tools

---

✨ Key Features

🌟 AI Agent Swarm Visualisation

• Real-time Hive Mind Display: Watch AI agents collaborate in stunning 3D
• Claude Flow Integration: Full MCP (Model Context Protocol) support for agent orchestration
• GPU-Accelerated Physics: Smooth 60 FPS visualisation of 200+ agents
• Interactive Control: Spawn, configure, and command agent swarms in real-time
• 12 Specialist Agent Types: Queen, Coordinator, Researcher, Coder, Analyst, and more

📊 Dual Graph Architecture

• Parallel Visualisation: View Logseq knowledge graphs and AI agent swarms simultaneously
• Independent Physics: Each graph has its own optimised physics engine
• Binary Protocol: 85% bandwidth reduction for massive scalability
• Cross-Graph Analytics: Compare and link knowledge with AI agent activities
• Unified Physics Engine: GPU-accelerated simulation for both graph types

🧠 Integrated AI Services

• Conversational Q&A: Chat with your knowledge graph using RAGFlow and Perplexity
• Voice Interaction: High-quality Text-to-Speech (TTS) and Speech-to-Text (STT)
• AI-Powered Analytics: Clustering, anomaly detection, and intelligent insights
• Semantic Analysis: Compare knowledge and agent graph structures

🚀 Production-Ready Architecture

• Rust Backend: High-performance Actix-based server with actor model
• React/Three.js Frontend: Immersive 3D visualisation with WebXR support
• Docker Deployment: One-command deployment with full MCP integration
• Enterprise Features: Authentication, monitoring, and horizontal scaling

---

🏗️ System Architecture

graph TD
    subgraph "Data Sources"
        LS["Logseq Graph<br/>(Markdown Files)"]
        GH["GitHub Repository<br/>(Knowledge Base)"]
        CF["Claude Flow MCP<br/>(AI Agents)"]
    end

    subgraph "Frontend (React/Three.js)"
        A["Dual Graph Renderer<br/>(Logseq + Agents)"]
        B["Settings Panel<br/>(Zustand Store)"]
        C["Analytics Controls"]
        D["Command Palette"]
        E["Conversation Pane<br/>(RAGFlow/Perplexity)"]
    end

    subgraph "Communication Layer"
        F["REST API (JSON)<br/>Control & Data"]
        G["WebSocket (Binary)<br/>Position & Velocity"]
    end

    B & C & D & E --> F
    A <--> G

    subgraph "Backend (Rust/Actix)"
        H["API Handlers<br/>/api/*"]
        I["Socket Handler<br/>/wss"]
        J["Graph Actor<br/>(Dual Graphs)"]
        K["Settings Actor"]
        L["Claude Flow Actor"]
        M["GPU Compute Actor"]
        N["Client Manager Actor"]
        O["GitHub Service<br/>(File Sync)"]
        P["RAGFlow Service"]
    end

    F --> H
    G <--> I
    H --> J & K & L & O & P
    I <--> J & N
    J --> M

    subgraph "External Services"
        Q["TCP Port 9500<br/>Claude Flow MCP"]
        R["CUDA<br/>GPU Physics"]
        S["OpenAI/Perplexity<br/>AI Services"]
    end

    LS --> GH
    GH --> O
    L --> Q
    M --> R
    P --> S
    CF --> Q

    style LS fill:#e8f5e9
    style GH fill:#e8f5e9
    style CF fill:#fff3e0
    style Q fill:#e3f2fd
    style R fill:#fce4ec

Loading

AI Services Integration

graph TD
    subgraph Client
        UI["ConversationPane (React)"]
    end

    subgraph "Backend (Rust)"
        APIServer["API Server (Actix)"]
        RAGFlowSvc["RAGFlowService.rs"]
        PerplexitySvc["PerplexityService.rs"]
        SpeechSvc["SpeechService.rs"]
    end

    subgraph "External AI APIs"
        RAGFlowAPI["RAGFlow API"]
        PerplexityAPI["Perplexity API"]
        OpenAI_API["OpenAI API (TTS/STT)"]
        KokoroAPI_Ext["Kokoro API (TTS)"]
    end

    UI -->|REST API Call| APIServer
    APIServer --> RAGFlowSvc
    APIServer --> PerplexitySvc
    APIServer --> SpeechSvc

    RAGFlowSvc --> RAGFlowAPI
    PerplexitySvc --> PerplexityAPI
    SpeechSvc --> OpenAI_API
    SpeechSvc --> KokoroAPI_Ext

    OpenAI_API -->|Audio/Text| SpeechSvc
    KokoroAPI_Ext -->|Audio| SpeechSvc
    RAGFlowAPI -->|Text| RAGFlowSvc
    PerplexityAPI -->|Text| PerplexitySvc

    RAGFlowSvc -->|Response| APIServer
    PerplexitySvc -->|Response| APIServer
    SpeechSvc -->|Response| APIServer

    APIServer -->|Response| UI

Loading

---

🛠️ Technology Stack

Backend

• Rust with Actix-web framework for high-performance concurrent processing
• CUDA for GPU-accelerated physics simulation
• TCP for direct MCP communication with Claude Flow
• WebSocket with binary protocol for real-time updates
• Actor Model for safe concurrent state management

Frontend

• React with TypeScript for type-safe component architecture
• Three.js for GPU-accelerated 3D rendering
• Zustand for centralised state management with single authoritative store
• WebXR for AR/VR capabilities including Quest 3 support
• Binary WebSocket Protocol for 85% bandwidth reduction

Infrastructure

• Docker with multi-stage builds for containerisation
• Nginx as reverse proxy with WebSocket support
• GitHub Actions for CI/CD pipeline

---

📦 Installation & Setup

System Requirements

• Docker: 20.10+ with Docker Compose
• Node.js: 20+ (for local development)
• Rust: 1.75+ (for backend development)
• GPU: NVIDIA GPU with CUDA support (optional but recommended)
• Browser: Chrome/Firefox/Safari with WebGL support

Configuration

Create and configure your environment file:

cp .env_template .env

Key environment variables:

# Claude Flow MCP (TCP)
CLAUDE_FLOW_HOST=multi-agent-container
MCP_TCP_PORT=9500
MCP_TRANSPORT=tcp

# API Keys
OPENAI_API_KEY=your_key
PERPLEXITY_API_KEY=your_key
GITHUB_TOKEN=your_token  # Required for Logseq graph sync

# Optional Features
NO_GPU_COMPUTE=false  # Set to true if no GPU available

Deployment Options

Production Deployment

docker-compose up -d

Development Environment

docker-compose -f docker-compose.dev.yml up

---

🔮 What's New in VisionFlow

AI Agent Features

• Hive Mind Spawning: Create coordinated AI agent swarms with different topologies (mesh, hierarchical, ring, star)
• Real-time Monitoring: Track agent performance, task completion, and communication patterns
• MCP Tool Integration: 70+ tools including task orchestration, memory management, and neural training
• Binary Protocol: 28-byte format reduces bandwidth by 85% compared to JSON

Technical Achievements

• GPU Physics: Process 200+ agents at 60 FPS with GPU-accelerated spring physics
• Actor Model: Rust backend using Actix actors for safe concurrent state management
• TCP Bridge: Clean separation between UI server and agent execution environment
• Dual Graph System: Simultaneous Logseq knowledge graph and AI agent visualisation

---

📊 Performance Benchmarks

Metric	Performance
Max Agents (60 FPS)	200+
Position Update Latency	< 5ms
Binary Protocol Efficiency	28 bytes/agent
Network Bandwidth (100 agents)	168 KB/s
Memory Usage	28 bytes/agent
GPU Acceleration	2.8-4.4x speed improvement

---

🎮 Advanced Features

Analytics Suite

• Clustering Algorithms: Spectral, DBSCAN, K-Means++, Louvain
• Anomaly Detection: Isolation Forest, LOF, Autoencoder
• Real-Time Insights: AI-generated recommendations
• Time-Travel: Navigate through graph state history

XR/AR Capabilities

• Quest 3 Integration: Native AR support with passthrough
• Hand Tracking: Natural interaction with joint position detection
• Spatial UI: 3D interface elements with gesture recognition
• Voice Controls: Audio commands and TTS responses

Authentication & Settings

• Nostr Integration: Decentralised authentication
• User-Specific Settings: Personalised configurations with cloud sync
• Power User Mode: Advanced features and global settings access
• Cross-Session Persistence: Maintain state across browser sessions

---

📚 Documentation Hub

Comprehensive documentation is now organised in the new docs/ structure:

• 📚 Complete Documentation - Main documentation hub with organised structure
• 📖 Guides - User and developer guides, tutorials
• 🔧 Technical Documentation - Implementation details and system specs
• 📚 Reference Documentation - API docs and configuration reference
• 🏗️ Architecture - System design and architectural decisions
• 📦 Archive - Historical documentation and legacy content

---

🧪 Testing & Development

Run Tests

# Rust backend tests
cargo test

# Frontend tests
cd client && npm test

# TCP connection test
cargo run --bin test-tcp

# Performance benchmarks
cargo bench

Development Workflow

1. Fork the repository
2. Create a feature branch
3. Make your changes following the style guide
4. Run tests and linting
5. Submit a pull request

---

🤝 Community & Support

• GitHub Issues: Report bugs or request features
• Discord: Join our community (Coming soon)
• Twitter: @VisionFlowAI (Coming soon)
• Documentation: Full Documentation

---

🙏 Acknowledgements

• Prof Rob Aspin: For inspiring the project's vision and providing valuable resources
• Claude Flow Team: For the amazing MCP hive-mind agent framework
• Anthropic: For the Model Context Protocol specification and Claude AI models
• OpenAI: For their advanced AI models powering the question-answering features
• Perplexity AI and RAGFlow: For their AI services enhancing content processing
• Three.js Community: For the robust 3D rendering capabilities
• Actix Team: For the high-performance web framework powering the backend
• Rust and React Communities: For building the foundation technologies

---

📄 Licence

This project is licensed under the Mozilla Public License 2.0. See the LICENSE file for details.

---

Built using AI-driven development

VisionFlow - Visualising the future of AI multi-agent systems 🚀

📚 Documentation

The complete documentation is organised in the new structured format:

• 📚 Documentation Hub - Central documentation index with organised structure
• 📖 Getting Started - Installation, quickstart, and configuration guides
• 🔧 Technical Details - Implementation details and system specifications
• 📚 API Reference - REST, WebSocket, and agent documentation
• 🏗️ Architecture - System design and architectural decisions

🛠️ Development Status

VisionFlow is in active development with regular updates to both functionality and documentation. The project showcases advanced multi-agent AI coordination with real-time visualisation capabilities.

Key Features

• Multi-Agent Orchestration: Coordinate swarms of AI agents with different topologies
• Real-time 3D Visualisation: GPU-accelerated rendering of agent behaviour
• Consensus Protocols: Multiple distributed consensus implementations
• Neural Pattern Recognition: AI-powered pattern detection and learning
• Comprehensive API: RESTful and WebSocket interfaces for integration

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on how to get involved.

📄 Licence

This project is licensed under the Mozilla Public License 2.0 - see the LICENSE file for details.

---