…on plan (#1492)

Records the next-bug-class finding from the M-GPU-MOE-1.3 partial
discharge (PR #1491 squash f0cbe37, MERGED 2026-05-04 on aprender
main).

WHAT 1.3 DISCHARGED:

  FALSIFY-QW3-MOE-GPU-PRELOAD-001 — wrapper construction now
  succeeds for qwen3_moe GGUFs.

WHAT 1.3 EXPOSED (NOT YET FIXED):

  Heavy `qwen3_moe_gpu_parity` test on lambda-vector RTX 4090 +
  cached 17.3 GB Qwen3-Coder GGUF runs end-to-end but fails:

    assert!(gpu_logits.iter().all(|v| v.is_finite()),
            "all GPU logits must be finite (no NaN/Inf)");

  Blocks FALSIFY-QW3-MOE-GPU-PARITY-001 + PARTIAL-discharges
  FALSIFY-QW3-MOE-GPU-INVARIANTS-001.

BISECTION PLAN (M-GPU-MOE-1.4):

  (a) Instrumentation — extend apr trace --json --payload to
      capture per-stage tensors on MoE GPU path
  (b) Bisection — diff CPU vs GPU per-stage to find first
      NaN/Inf-producing stage. Candidates:
        * Q4K matvec accumulator overflow
        * SwiGLU silu Inf
        * top-k router renorm div-by-zero
        * missing per-head Q/K RMSNorm in MoE GPU path
  (c) Fix — apply at bisected stage; class TBD by result

THIS PR ADDS:

  * v1.3.0 → v1.4.0 amendment_history block (~110 lines)
  * NEW M-GPU-MOE-1.4 implementation_stage (PENDING)
  * M-GPU-MOE-1.3 status updated PENDING → PARTIALLY_DISCHARGED
  * Top-level version + status block updated

VALIDATION: pv validate → 0 errors, 0 warnings.

Per CLAUDE.md "NEVER write code before writing a provable contract"
— this PR pins the bisection-and-fix plan BEFORE code. Code
follows in M-GPU-MOE-1.4 fix PR (separate scope).

Refs: M52, M53, M54, M55, M56, R10,
      qwen3-moe-forward-gpu-v1 v1.4.0,
      FALSIFY-QW3-MOE-GPU-INVARIANTS-001 (finiteness sub-check).

Co-authored-by: Claude Opus 4.7 <[email protected]>

Sibling contract to trace-attn-sub-stages-v1; pins the SaveTensorStage
extensions needed for M-GPU-MOE-1.4 NaN/Inf bisection per
qwen3-moe-forward-gpu-v1 v1.4.0 amendment.

WHY THIS CONTRACT:

  After M-GPU-MOE-1.3 partial fix (PR #1491 squash f0cbe37
  MERGED), heavy `qwen3_moe_gpu_parity` test on lambda-vector
  RTX 4090 produces 100% NaN logits. Steps 1-9 are CPU-shared
  with finite output; step 10 (GPU MoE FFN — moe_ffn_forward_layer_cuda
  → expert_swiglu_cuda → q4k_matvec/q6k_gemv) is the only candidate.
  Per qwen3-moe-forward-gpu-v1 v1.4.0 bisection plan step (a),
  apr trace needs new SaveTensorStage variants for MoE GPU stages.

WHAT IT DEFINES:

  * 2 mandatory new SaveTensorStage variants:
    - MoeRouter — top-k weights post-softmax/renormalize
    - MoeFfnOut — aggregated MoE FFN output (Σ w_e * expert_out_e)
  * 4 optional per-expert variants (with expert_id qualifier):
    - MoeExpertGate, MoeExpertUp, MoeExpertSwigl, MoeExpertOut
    - Promoted to mandatory only if 2-stage bisection isn't
      precise enough.

  * Bisection chain CPU-vs-GPU:
    cos_sequence = [
      cos(CPU.ffn_norm,    GPU.ffn_norm),     # parent enum
      cos(CPU.moe_router,  GPU.moe_router),   # NEW
      cos(CPU.moe_ffn_out, GPU.moe_ffn_out),  # NEW
    ]

FALSIFICATION TESTS (4):

  FALSIFY-MOE-SUB-001: New variants exist + parse correctly
  FALSIFY-MOE-SUB-002: Existing 20-stage byte-identity preserved
  FALSIFY-MOE-SUB-003: Bisection identifies first NaN-producing stage
  FALSIFY-MOE-SUB-004: Fix PR cites bisected stage by name

IMPLEMENTATION_STAGES (4):

  M-MOE-SUB-0: This contract scaffold (SHIPPED)
  M-MOE-SUB-1: Add MoeRouter + MoeFfnOut variants (PENDING)
  M-MOE-SUB-2: Wire MoeRouter into both CPU + GPU forward (PENDING)
  M-MOE-SUB-3: Wire MoeFfnOut + run heavy bisection (PENDING)
  M-MOE-SUB-4: OPTIONAL per-expert promotion (PENDING)

VALIDATION: pv validate exits 0 errors, 0 warnings.

Per CLAUDE.md "NEVER write code before writing a provable contract"
— this PR pins the trace-stage architecture before code lands.
M-MOE-SUB-1 follows in a separate PR.

Refs: M-GPU-MOE-1.4, R10, qwen3-moe-forward-gpu-v1 v1.4.0,
      trace-attn-sub-stages-v1 (sibling pattern).

Co-Authored-By: Claude Opus 4.7 <[email protected]>

Sibling contract to trace-attn-sub-stages-v1; pins the SaveTensorStage
extensions needed for M-GPU-MOE-1.4 NaN/Inf bisection per
qwen3-moe-forward-gpu-v1 v1.4.0 amendment.

WHY THIS CONTRACT:

  After M-GPU-MOE-1.3 partial fix (PR #1491 squash f0cbe37
  MERGED), heavy `qwen3_moe_gpu_parity` test on lambda-vector
  RTX 4090 produces 100% NaN logits. Steps 1-9 are CPU-shared
  with finite output; step 10 (GPU MoE FFN — moe_ffn_forward_layer_cuda
  → expert_swiglu_cuda → q4k_matvec/q6k_gemv) is the only candidate.
  Per qwen3-moe-forward-gpu-v1 v1.4.0 bisection plan step (a),
  apr trace needs new SaveTensorStage variants for MoE GPU stages.

WHAT IT DEFINES:

  * 2 mandatory new SaveTensorStage variants:
    - MoeRouter — top-k weights post-softmax/renormalize
    - MoeFfnOut — aggregated MoE FFN output (Σ w_e * expert_out_e)
  * 4 optional per-expert variants (with expert_id qualifier):
    - MoeExpertGate, MoeExpertUp, MoeExpertSwigl, MoeExpertOut
    - Promoted to mandatory only if 2-stage bisection isn't
      precise enough.

  * Bisection chain CPU-vs-GPU:
    cos_sequence = [
      cos(CPU.ffn_norm,    GPU.ffn_norm),     # parent enum
      cos(CPU.moe_router,  GPU.moe_router),   # NEW
      cos(CPU.moe_ffn_out, GPU.moe_ffn_out),  # NEW
    ]

FALSIFICATION TESTS (4):

  FALSIFY-MOE-SUB-001: New variants exist + parse correctly
  FALSIFY-MOE-SUB-002: Existing 20-stage byte-identity preserved
  FALSIFY-MOE-SUB-003: Bisection identifies first NaN-producing stage
  FALSIFY-MOE-SUB-004: Fix PR cites bisected stage by name

IMPLEMENTATION_STAGES (4):

  M-MOE-SUB-0: This contract scaffold (SHIPPED)
  M-MOE-SUB-1: Add MoeRouter + MoeFfnOut variants (PENDING)
  M-MOE-SUB-2: Wire MoeRouter into both CPU + GPU forward (PENDING)
  M-MOE-SUB-3: Wire MoeFfnOut + run heavy bisection (PENDING)
  M-MOE-SUB-4: OPTIONAL per-expert promotion (PENDING)

VALIDATION: pv validate exits 0 errors, 0 warnings.

Per CLAUDE.md "NEVER write code before writing a provable contract"
— this PR pins the trace-stage architecture before code lands.
M-MOE-SUB-1 follows in a separate PR.

Refs: M-GPU-MOE-1.4, R10, qwen3-moe-forward-gpu-v1 v1.4.0,
      trace-attn-sub-stages-v1 (sibling pattern).

Co-authored-by: Claude Opus 4.7 <[email protected]>