Thanks to visit codestin.com
Credit goes to github.com

Skip to content

lmvdz/pad

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Price Anomaly Detection (PAD)

Real‑time crypto price anomaly detection with a dual‑stream frontend, a high‑performance WebSocket backend, and NATS‑based pub/sub. The system powers per‑symbol alerting for fast trade actions and market‑wide intelligence (stress, synchronized moves, sector anomalies). A TealStreet‑ready UI component is included.

Highlights

  • Real‑time anomaly alerts (per‑symbol) + market‑wide anomalies in parallel
  • Single WebSocket endpoint with channel subscriptions (alerts, market)
  • Backward‑compatible per‑path endpoints (/ws/alerts, /ws/market)
  • Influx/HTTP history feeds for backfill in the UI
  • TealStreet integration: all UI logic consolidated in a single component.tsx

Repository Layout

  • packages/backend — uWebSockets.js server, NATS integration, HTTP APIs, test tools
  • packages/frontend — TealStreet iterator UI (single‑file component), utilities

Architecture

Data flow (publish → transform → stream → render):

  1. Producers publish alerts/anomalies to NATS
  2. Backend subscribes and broadcasts via WebSocket
  3. Frontend connects once to /ws and subscribes to channels
  4. UI renders per‑symbol Alerts list and Market dashboard

Channels

We use one WebSocket connection and “channels” to multiplex data types:

  • alerts — per‑symbol anomalies (used by Alerts list)
  • market — market‑wide anomalies (used by Market dashboard)

Clients subscribe/update filters by sending JSON:

{
  "type": "subscribe",
  "filters": {
    "symbols": ["BTCUSDT", "ETHUSDT"],
    "methods": ["Z_SCORE"],
    "severities": ["HIGH", "CRITICAL"],
    "channels": ["alerts", "market"]
  }
}

WebSocket Endpoints

  • Recommended: ws://localhost:3001/ws (single connection, channel‑aware)
  • Backward compatible:
    • ws://localhost:3001/ws/alerts
    • ws://localhost:3001/ws/market

HTTP Endpoints

  • GET /alerts — recent per‑symbol alerts (JSON)
  • GET /market — recent market‑wide anomalies (JSON)

Getting Started

Prerequisites

  • Node.js 18+
  • pnpm or npm
  • NATS server (required for real-time features and WebSocket functionality)

Setup

  1. Install NATS server (see detailed instructions in packages/backend/README.md):

    # macOS
    brew install nats-io/nats/nats-server
    
    # Linux - download from https://github.com/nats-io/nats-server/releases
    # Or use Go: go install github.com/nats-io/nats-server/v2@latest
  2. Configure environment:

    cp packages/backend/.env.example packages/backend/.env
    # Edit .env with your settings

    Key variables:

    • NATS_URL (default nats://localhost:4222)
    • INFLUXDB_URL, API_TOKEN, DATABASE (for data persistence)

Development Workflow

For a complete development setup, start services in this order (each in a separate terminal):

# 1. Start InfluxDB
cd packages/backend
pnpm run start:influx

# 2. Start NATS server
pnpm run start:nats

# 3. Start frontend (in dev mode)
cd packages/frontend
pnpm run dev

# 4. Start data ingestion
cd packages/backend
pnpm run ingestion

# 5. Start analytics
cd packages/backend
pnpm run analytics

# 6. Start WebSocket alerts server
cd packages/backend
pnpm run ws:alerts

The WebSocket server listens at http://localhost:3001 and ws://localhost:3001/ws.

Run the frontend (TealStreet iterator)

The TealStreet component is authored in a single file for copy‑paste into TealStreet:

packages/frontend/src/tealstreet-iterator/component.tsx

The local iterator dev app rebuilds on save. When integrating with TealStreet, use the produced single‑file component (a.k.a. component-ready.tsx if present/required by your workflow) and paste into TealStreet.

Developer Tools

Send quick demo market data

Publishes a short, curated sequence of market‑wide anomalies to NATS. Great for manual UI checks.

cd packages/backend
npx tsx src/tests/sendTestMarketData.ts

Generate streaming demo market data

Produces randomized market anomalies at an interval.

cd packages/backend
npx tsx src/tests/generateTestMarketData.ts [count] [intervalMs]
# e.g. 10 anomalies, every 3000ms
npx tsx src/tests/generateTestMarketData.ts 10 3000

Frontend Notes (TealStreet‑ready)

  • All logic (alerts list + market dashboard + connection + filtering) is consolidated in component.tsx to meet TealStreet’s single‑file requirement (except external global-module and api-types which are provided by TealStreet).
  • The UI uses a PAD WebSocket singleton and subscribes to channels on the single /ws endpoint to avoid multiple sockets.
  • Market anomalies are normalized defensively so the UI remains stable even when publishers differ slightly in shape.

Backend Notes

  • uWebSockets.js for high‑throughput WS/HTTP
  • NATS for pub/sub from analytics producers
  • Channel‑aware broadcast: messages tagged to alerts or market honoring client filters
  • Legacy compatibility: /ws/alerts, /ws/market remain available

Data Model (high level)

  • Per‑symbol alert (alerts channel): id, symbol, severity, method, scores/timestamps
  • Market anomaly (market channel):
    • SYNCHRONIZED_MOVEMENT — direction, synchronization ratio, affected symbols
    • MARKET_STRESS — metric (volatility/breadth), magnitude
    • SECTOR_ANOMALY — sector, performance/deviation, affected symbols

Roadmap

  • Enrich sector mapping via CoinGecko categories with freshness checks
  • Persist alerts into a dedicated table for historical queries while streaming live via WS
  • Expand UI layouts (multiple list item layouts + header dropdown)
  • Add market stress summary panel and actions

Troubleshooting

  • If the browser can’t connect to WS, confirm the server logs show:
    • 🔊 uWebSockets.js alert server listening on ws://0.0.0.0:3001/ws
  • If alerts stop after changing filters, ensure channels are still included in your subscribe/update_filters messages.
  • To validate WS endpoints quickly, use the provided src/tests/testChannelWebSocket.ts or testMarketWebSocket.ts utilities (run with npx tsx).

Maintained by @lmvdz. Contributions welcome.

About

Price Anomaly Detection

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published