verification

Ruflo Verification System

A cross-platform, cryptographically-signed regression-protection toolkit. Every documented fix in ruflo gets attested by a SHA-256 + marker substring + Ed25519 signature. Anyone with the same git commit can re-derive the public key and verify it independently.

Three-layer regression protection:

Behavioral smoke tests (CI) — exercise user-visible failure modes

Cryptographic witness manifest (this folder) — attest fix presence

Temporal history (<os>/history.jsonl) — bisect when a regression was introduced

See docs/validation/README.md for the full architecture, ADR-102 for the smoke-harness pattern, and ADR-103 for the temporal-history layer.

Why this exists

Three regressions filed on 2026-05-08 (#1859, #1862, #1867) all passed unit tests on the broken commits but broke for users on first install. Unit tests verify code paths; users hit subprocess flag parsers, fresh npm install resolution, plugin/CLI version drift. The witness layer attests that every documented fix's load-bearing code is still present — even when the fix has no dedicated smoke test.

A regression that deletes the load-bearing line of a fix flips its markerVerified to false. The CI's witness-verify job blocks publish.

Folder layout

verification/
  README.md                  ← you are here
  witness-fixes.json         ← input config (OS-independent)
  mcp-tool-baseline.json     ← ADR-112 — monotone-decreasing tool-discoverability baseline
  results.md                 ← human-readable last-run report
  inventory.json             ← reserved for capability inventory
  linux/
    manifest.md.json         ← signed by Linux CI runner
    history.jsonl            ← Linux temporal log
  macos/
    manifest.md.json         ← signed by macOS CI runner
    history.jsonl            ← macOS temporal log
  windows/
    manifest.md.json         ← signed by Windows CI runner
    history.jsonl            ← Windows temporal log

Why per-OS? File hashes drift across platforms (LF vs CRLF normalization, path separators, prebuilt binary differences). Each OS produces its own truthful snapshot. The witness-verify CI job runs once per OS and writes/verifies that OS's bundle, catching platform-specific marker drift before it reaches users on that platform.

Why JSONL inside each OS folder? The witness use case is structured key-value retrieval (snapshot N's commit + fix statuses), not vector similarity search. JSONL is git-diff-friendly, append-cheap, and human-readable — the better fit. The folder layout adopts the RVF-style cognitive container concept (per-OS bundle of manifest + history + metadata); the file format inside is JSONL.

Tutorial — adopt the witness in your own project

1. Bootstrap

The witness toolkit lives in plugins/ruflo-core/scripts/witness/ — copy it into your project (or install the ruflo-core plugin).

# Initialize per-OS folders + sample fix template
node plugins/ruflo-core/scripts/witness/init.mjs --root .

# Single runtime dependency
npm i @noble/ed25519

Outputs:

verification/
  witness-fixes.json     # template — edit with your fixes
  <os>/
    manifest.md.json     # empty seed
    history.jsonl        # empty seed

2. Register your first fix

Open verification/witness-fixes.json:

{
  "fixes": [
    {
      "id": "MY-001",
      "desc": "Race condition in token refresh — early-return on in-flight refresh",
      "file": "dist/auth.js",
      "marker": "if (this._refreshing) return this._refreshing;"
    }
  ]
}

Choosing a marker is the load-bearing skill:

Bad marker	Why it fails
`'function'`	False-positives in any file with a function
`'TODO'`	Flaps as TODOs come and go
`'fix'`	Too generic

Good marker	Why it works
`(await import('better-sqlite3')).default`	Distinctive AND specific to the fix mechanism
`(ctx.flags.file as string) \|\| ctx.args[0]`	The exact swap that fixes the bug
`import * as bcrypt from 'bcryptjs'`	Proves the bcrypt → bcryptjs migration is in dist

Pick a unique substring from the diff that's specifically created by the fix.

3. Generate the signed manifest

In ruflo's monorepo there's a wrapper that hard-codes paths:

node scripts/regen-witness.mjs

For external projects, call the plugin script with explicit paths:

node plugins/ruflo-core/scripts/witness/regen.mjs \
  --manifest verification/macos/manifest.md.json \
  --history  verification/macos/history.jsonl \
  --fixes    verification/witness-fixes.json

Output:

witness regen summary
─────────────────────
  gitCommit:    abc123def456…
  branch:       main
  issuedAt:     2026-05-09T00:00:00.000Z
  total fixes:  1  (was 0)
  verified:     1
  missing:      0
  new entries:  MY-001
written:  verification/macos/manifest.md.json
appended: verification/macos/history.jsonl

verified: N/N confirms every marker is present in its cited file.

4. Verify

node plugins/ruflo-core/scripts/witness/verify.mjs \
  --manifest verification/macos/manifest.md.json

Output:

Manifest signature:
  hash matches:                    yes
  public key reproducible:         yes
  Ed25519 signature valid:         yes

Summary: pass=1 drift=0 regressed=0 missing=0

If regressed > 0 or signatureValid: no, the manifest has been tampered with or a documented fix's marker has been deleted.

5. Investigate a regression

When CI reports F12 regressed:

node plugins/ruflo-core/scripts/witness/history.mjs \
  --history verification/macos/history.jsonl regressions

Output:

F12
  last pass:    a1b2c3d4  2026-05-07T14:23:11.000Z
  regressed at: 9f8e7d6c  2026-05-08T09:14:55.000Z

Then git log lastPass..regressedAt -- <file> collapses bisect from "many commits" to "the few in this 18-hour window."

6. Wire into CI

Add to .github/workflows/<your-pipeline>.yml:

witness-verify:
  strategy:
    matrix:
      os: [ubuntu-latest, macos-latest, windows-latest]
  runs-on: ${{ matrix.os }}
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with: { node-version: '22' }
    - run: npm i @noble/ed25519
    - shell: bash
      run: |
        case "$RUNNER_OS" in
          Linux)   OS_DIR=linux ;;
          macOS)   OS_DIR=macos ;;
          Windows) OS_DIR=windows ;;
        esac
        node plugins/ruflo-core/scripts/witness/verify.mjs \
          --manifest "verification/$OS_DIR/manifest.md.json" \
          --json > /tmp/result.json
        node -e "if (!require('/tmp/result.json').ok) process.exit(1)"

Block publish on failure. Every commit on every OS gets attested.

Features

Feature	Mechanism	What it catches
SHA-256 fingerprint per fix	Per-file hash recorded at issuance	Any change to the file
Marker substring	Distinctive code substring required to be present	Refactors that delete the fix
Ed25519 signature	Deterministic seed `sha256(gitCommit + ':ruflo-witness/v1')`	Manifest tampering, key-derivation drift
Reproducible public key	Anyone with the git commit re-derives the same pubkey	No committed private key needed
Per-OS bundles	`verification/{linux,macos,windows}/`	Platform-specific marker drift (CRLF, path separators)
Temporal history (JSONL)	One snapshot appended per regen	When a regression was introduced
Drift vs regression	sha256 changed but marker present → drift; marker missing → regression	False alarms suppressed; real regressions flagged
Status timeline per fix	`history.mjs timeline --id F12`	Flapping markers (too brittle) and persistent drift (too unstable)

Daily workflow cheat sheet

# Add a new fix
$EDITOR verification/witness-fixes.json
node scripts/regen-witness.mjs

# Verify the current tree
node plugins/ruflo-core/scripts/witness/verify.mjs \
  --manifest verification/macos/manifest.md.json

# Diff against the previous snapshot
node plugins/ruflo-core/scripts/witness/history.mjs \
  --history verification/macos/history.jsonl summary

# Find when a regression was introduced
node plugins/ruflo-core/scripts/witness/history.mjs \
  --history verification/macos/history.jsonl regressions

# Status timeline for one fix
node plugins/ruflo-core/scripts/witness/history.mjs \
  --history verification/macos/history.jsonl timeline --id F12

Schema

`<os>/manifest.md.json`

{
  "manifest": {
    "schema": "ruflo-witness/v1",
    "issuedAt": "2026-05-09T00:00:00.000Z",
    "gitCommit": "<full sha>",
    "branch": "main",
    "os": "macos",                              // matches the folder
    "releases": { "ruflo": "3.7.0-alpha.21", "@claude-flow/plugin-agent-federation": "1.0.0-alpha.15" },
    "summary": { "totalFixes": 102, "verified": 102, "missing": 0 },
    "fixes": [
      {
        "id": "F1",
        "desc": "...",
        "file": "v3/@claude-flow/cli/dist/...",
        "sha256": "<64 hex>",
        "marker": "<distinctive substring>",
        "markerVerified": true
      }
    ]
  },
  "integrity": {
    "manifestHashAlgo": "sha256",
    "manifestHash": "<64 hex of canonical manifest>",
    "signature": "<64-byte hex of Ed25519 signature over canonical manifest>",
    "signatureAlgo": "ed25519",
    "publicKey": "<32-byte hex>",
    "signature": "<64-byte hex>",
    "seedDerivation": "sha256(gitCommit + ':ruflo-witness/v1')"
  }
}

`<os>/history.jsonl` (one entry per line)

{
  "v": 1,
  "commit": "<full sha>",
  "issuedAt": "2026-05-09T00:00:00.000Z",
  "branch": "main",
  "os": "macos",
  "manifestHash": "<hex>",
  "summary": { "totalFixes": 82, "verified": 82, "missing": 0 },
  "fixes": {
    "F1":     { "sha256": "...", "markerVerified": true },
    "#1867":  { "sha256": "...", "markerVerified": true }
  }
}

`witness-fixes.json` (input, OS-independent)

{
  "fixes": [
    {
      "id": "MY-001",
      "desc": "human-readable description",
      "file": "path/relative/to/repo/root.js",
      "marker": "distinctive substring that must remain in the file"
    }
  ]
}

Anti-patterns

Hand-editing manifest.md.json — breaks the signature. Always regenerate via regen.mjs.
Markers that match generic patterns ('function', 'import', 'TODO') — false positives or flapping.
Skipping the history append — without --history, you lose Layer 3 (regression introduction time).
Committing the manifest without the history — they belong in the same commit.
Cross-OS manifest editing — the manifest is signed for the OS that generated it. Don't copy a Linux manifest into the macOS folder.

How it integrates with the rest of ruflo's regression stack

Layer	Where	What it catches
Layer 1 — install smoke	`v3/@claude-flow/memory/scripts/smoke-no-bsqlite.mjs` + CI `smoke-install-no-bsqlite`	`npm install` failures on platforms without prebuilds
Layer 1 — hook smoke	`plugins/ruflo-core/scripts/test-hooks.mjs` + CI `plugin-hooks-smoke`	Plugin/CLI flag drift, parser ambiguity
Layer 1 — MCP protocol smoke	`plugins/ruflo-core/scripts/test-mcp-protocol.mjs` + CI `mcp-protocol-smoke`	HTTP MCP wire-format compliance
Layer 1 — memory-import smoke	`plugins/ruflo-core/scripts/test-memory-import.mjs` + CI `memory-import-smoke`	Memory_import_claude WSL path + key sanitization regressions
Layer 1 — tool-description audit	`scripts/audit-tool-descriptions.mjs` + CI `tool-descriptions-audit` (ADR-112)	Every MCP tool description must have "Use when … is wrong because …" guidance + length ≥ 80 + unique. Baseline at `verification/mcp-tool-baseline.json` monotone-decreasing
Layer 2 — witness manifest	`verification/<os>/manifest.md.json` + CI `witness-verify`	Documented fix marker disappearing
Layer 3 — temporal history	`verification/<os>/history.jsonl` + `history.mjs`	When a regression was introduced

All CI jobs gate publish. A regression in any layer blocks the release on the platform where it was caught.

References

docs/validation/README.md — full architecture overview (in-repo)
Public gist — external-friendly version
ADR-102 — smoke harness pattern + flag-priority CLI convention
ADR-103 — JSONL history layer + plugin distribution
plugins/ruflo-core/skills/witness/SKILL.md — Claude Code skill workflow
plugins/ruflo-core/agents/witness-curator.md — curator agent definition
.github/workflows/v3-ci.yml — smoke-install-no-bsqlite, plugin-hooks-smoke, witness-verify jobs

Name		Name	Last commit message	Last commit date
parent directory ..
linux		linux
macos		macos
windows		windows
CAPABILITIES.md		CAPABILITIES.md
README.md		README.md
cli-mcp-tool-baseline.json		cli-mcp-tool-baseline.json
inventory.json		inventory.json
mcp-tool-baseline.json		mcp-tool-baseline.json
results.md		results.md
witness-fixes.json		witness-fixes.json

FilesExpand file tree

verification

Directory actions

More options

Directory actions

More options

Latest commit

History

verification

Folders and files

parent directory

README.md

Ruflo Verification System

Why this exists

Folder layout

Tutorial — adopt the witness in your own project

1. Bootstrap

2. Register your first fix

3. Generate the signed manifest

4. Verify

5. Investigate a regression

6. Wire into CI

Features

Daily workflow cheat sheet

Schema

<os>/manifest.md.json

<os>/history.jsonl (one entry per line)

witness-fixes.json (input, OS-independent)

Anti-patterns

How it integrates with the rest of ruflo's regression stack

References

`<os>/manifest.md.json`

`<os>/history.jsonl` (one entry per line)

`witness-fixes.json` (input, OS-independent)