Call for probes: building the GLM-5.2 Expert Atlas — measuring what each of the 19,456 experts actually does #175

The idea

The dashboard's new Brain page (#126) draws all 19,456 experts as a live cortex — tier, routing heat, per-turn firing. The obvious next question, and the one everybody asks first when they see it: what does this expert do?

Nobody knows — not us, not the model card. MoE experts differentiate on their own during training. But it's measurable, the engine already has everything needed, and this is a call for help doing it properly.

Method (no engine changes needed)

1. Run a batch of prompts from one topic category (say, code)
2. Diff .coli_usage snapshots before/after (STATS dump is already in the engine) → that category's expert-activation spectrum
3. Repeat per category → every expert gets a topic-affinity vector
4. Low-entropy experts are specialists ("code: 47%"), high-entropy ones are generalists
5. Ship the result as a static experts.json — the Brain page tooltip shows it on hover, and anyone can analyse it further

One run on a full-residency rig takes under an hour. The artifact is model-wide and permanent: an atlas of GLM-5.2's expert specialisation, which as far as we know nobody has published for any 700B-class MoE.

What we're contributing

We'll run the first sweep on the 6×5090 full-residency machine (10 categories × 3 prompts × 64 tokens) and publish the raw diffs + the aggregated atlas here.

Where you come in

• Probe design — the atlas is only as good as the probe set. Propose categories and prompts: languages (中文/日本語/Deutsch/…), domains (law, medicine, music theory), formats (JSON, SQL, regex), styles (poetry, formal, casual). 3–5 prompts per category, diverse phrasing.
• Cross-validation — run the same probe set on your hardware (any tier config works, only routing matters) and compare: expert specialisation should be machine-independent. Confirming that is itself a result.
• Co-activation analysis — @locallypwned's routing-trace work in Profile-guided expert layout + co-activation profiles — data from a GGUF-side experiment (mbolt) #119 is the complementary axis: which experts fire together. Topic affinity × co-activation communities = functional brain regions.
• Statistics — better specialisation metrics than entropy-argmax are welcome (tf-idf over topics? chi-squared vs the base rate?).

If this works, every colibrì user hovering the Brain page gets to see what part of a 744B mind lights up — and why.