Build smaller specialist models from one compatible Mixture of Experts model, test what they retain, and use MetaHarness to choose a specialist for each request.
Published interactive explainer · Usage guide · Architecture decisions · Evidence and limits
A Mixture of Experts, or MoE, model contains many expert networks. It activates a few of them at each layer while answering a request. MoE Foundry measures which experts a workload uses, keeps a selected group in each layer, and exports a smaller model for testing.
Think of one large team becoming several focused teams. Each team still needs the shared language machinery: embeddings, attention, normalization, tokenizer, and output head. One expert by itself is not a complete language model. A dense model without MoE experts needs a different distillation or training approach.
The resulting mixture of expert mixtures has two routing levels. MetaHarness chooses a complete specialist for the request. That specialist's original token router chooses experts inside the model. The full parent remains the fallback. LatentMesh integration starts with explicit task state and integrity checks; raw hidden state transfer and cross specialist KV reuse stay disabled.
The business question is measurable: can a specialist complete the same useful tasks with less memory and acceptable latency? Smaller checkpoint files alone do not answer it. Loading every specialist plus its parent can increase total fleet memory.
| Component | Implemented behavior | Current boundary |
|---|---|---|
| Python separator | Inspect supported safetensors, select experts from calibration traces, export complete specialists, and build mixture manifests | Structural support for explicit Qwen2 MoE, Qwen3 MoE, and Mixtral floating layouts; real model capability remains unqualified |
| MetaHarness routing | Actual @metaharness/router 0.4.0 with validated examples, costs, quality threshold, and parent fallback |
Chooses a model ID; does not launch an inference server |
| MetaHarness CLI | Generated upstream MetaHarness 0.4.16 harness with Codex host adapter, Foundry commands, and MCP bridge | Local orchestration scaffold; no inference worker is started |
| Foundry CLI | Run fixtures, route requests, inspect config metadata, and invoke the local Python engine | No implicit model download, cloud job, or GPU allocation |
| MCP server | Seven tools for status, fixtures, routing, state deltas, metric gates, and bounded config inspection | Local stdio only; no shell, export, GPU, or network execution tool |
| State exchange | Typed JSON deltas with identity binding, ordering, size limits, and SHA256 integrity checks | Local application contract; not authenticated transport or LatentMesh LMS1 wire compatibility |
| Interactive Site | Plain language introduction, six scene Hyperframes explainer, workbench, and downloadable evidence | Synthetic browser demonstrations; model weights run locally through the engine |
Use Node.js 20 or newer. Python 3.12 matches the recorded engine environment. Run these commands from the repository root:
git clone https://github.com/ruvnet/MoE-Foundry.git
cd MoE-Foundry
npm ci
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r engine/requirements.lock
python -m pip install --no-deps -e engine
npm run cli -- status
npm run cli -- demo --seed 42 --retain 0.6
npm run cli -- route --input examples/route.json
npm run cli -- inspect --root examples --path checkpoint/config.json
npm run acceptance
python -m pytest -q engine/testsThe demo requires no model weights or GPU. It reports results from a generated fixture and cannot authorize a production specialist. On Windows, activate the environment with .venv\Scripts\Activate.ps1 in PowerShell.
The repository includes a real scaffold generated with upstream [email protected], plus its kernel and Codex host adapter. Install its locked dependencies and run the Foundry bridge:
npm ci --prefix harness
npm run doctor --prefix harness
node harness/bin/cli.js foundry status
node harness/bin/cli.js foundry demo --seed 42 --retain 0.6
node harness/bin/cli.js foundry route --input examples/route.jsonThis orchestration CLI is separate from the @metaharness/router package that chooses specialist IDs. Both use the local Foundry contracts. See the harness guide for setup and the MCP bridge.
Supply a licensed local checkpoint and a calibration trace. First inspect the checkpoint:
moe-separator inspect /models/parent --metadata-onlyThen select, export, and compose. The example below retains 64 experts per routed layer; choose a count between the inspected token top K and the parent's expert count.
mkdir -p runs
moe-separator select /models/parent calibration.jsonl \
--retain 64 --method mass --out runs/selection.json
moe-separator export /models/parent runs/selection.json \
--domain code --out /models/code-specialist
moe-separator mixture /models/code-specialist/separator_receipt.json \
--fallback-id PARENT_ID --out runs/mixture.jsonReplace PARENT_ID with the parent_id in the selection or export receipt. Add more receipt paths to include more specialists. Exported specialists start unevaluated and disabled. The usage guide covers trace collection, multiple domains, the Node wrapper, and the required evaluation sequence.
Weight separation streams on CPU using a 1 MiB copy buffer by default. Calibration and generation benchmarks may need a GPU large enough for the chosen parent. No real GPU latency, memory reduction, or language model quality result is claimed in this release.
After installation, configure your client's local stdio MCP server. Replace both absolute paths with your checkout and the directory whose config files the client may inspect:
{
"mcpServers": {
"moe-foundry": {
"command": "node",
"args": [
"/absolute/path/MoE-Foundry/bin/moe-foundry.mjs",
"mcp",
"--root",
"/absolute/path/checkpoints"
]
}
}
}Tools: foundry_status, run_fixture, route_mixture, create_state_delta, apply_state_delta, evaluate_gate, and inspect_config. The status resource is ruv://moe-foundry/status. Inspecting config metadata does not validate tensor files. See MCP operations and security for the boundary and example calls.
The repository contains raw Python fixture results, export measurements, and browser/router measurements. These measure different operations and must be interpreted separately.
Publication checks passed: 37 JavaScript tests, 26 Python tests, 6 MetaHarness harness tests, and 13 CLI/MCP acceptance checks. The validation receipt records the scope, dependency hashes, and remaining qualification gates.
- Six persisted sharded export cases cover three model families and two floating tensor layouts. Structural tests verify expert remapping and shared tensor preservation using tiny synthetic checkpoints.
- Across five NumPy classification seeds, selected specialists matched the parent. Random pruning performed better in one seed. The fixture deliberately plants domain specialization and gives its request router the true domain; it is not an LLM benchmark.
- A synthetic 44.4 MiB checkpoint export measured a 0.1392 second median with the 1 MiB buffer. Increasing the buffer to 4 MiB improved the recorded median by only 1.6% and used more process memory. This measures export and integrity work, not inference speed.
Run the benchmarks locally:
npm run benchmark
python engine/examples/run_smoke.py
python engine/examples/benchmark_export.pyThese commands regenerate their result files. Compare measurements only under the same hardware, inputs, cache conditions, and runtime versions.
| Path | Purpose |
|---|---|
docs/usage.md |
Installation, CLI, calibration, export, routing, MCP, and qualification |
docs/adr/README.md |
All architecture decisions, including the CLI and MCP boundary |
docs/integrations.md |
Source pinned MetaHarness, ruvLLM, LatentMesh, Ruflo, and Core Memory contracts |
engine/ |
Installable Python separator and bounded local calibration adapter |
bin/ |
Foundry CLI and local MCP entry point |
src/ |
Shared validated contracts, metadata inspection, routing service, and MCP implementation |
harness/ |
Generated MetaHarness CLI, Codex host adapter, and Foundry bridge |
examples/ |
Routing inputs and client configuration examples |
dist/ |
Deployable interactive explainer and browser modules |
tests/ |
JavaScript behavior, interface, and safety checks |
docs/security-review.md |
Recorded review scope, controls, and remaining risks |
Native ruvLLM checkpoint loading, authenticated LatentMesh networking, Core Memory federation, and signed RVF delivery remain integration gates. Existing third party license notices are preserved with their code. Model weights and their licenses remain the operator's responsibility.
Start with a supported parent and export it while keeping every expert. Require the exported model to load in the intended runtime and reproduce parent logits before testing pruning. Then compare 80% retention against the parent and five equally sized random masks on at least 1,000 untouched tasks.
The proposed promotion gate requires at most 2 percentage points of quality loss, at least 30% lower measured peak device memory, and at most 10% worse p95 latency, with selected masks beating the random control and passing mixed domain tasks. Stop immediately if the full retention export changes logits; otherwise stop at a failed quality gate. Human review controls promotion. The largest unresolved risk is loss of real model capability after pruning; this test is the next decision gate.