feat: commitment-bound quote pricing (ADR-0003)#149
Open
grumbach wants to merge 3 commits into
Open
Conversation
Squashed foundation for the gossip-triggered contiguous-subtree audit (ADR-0002). Introduces the v12 storage-bound audit primitives the ADR work builds on: signed StorageCommitment + Merkle tree, recent_provers holder-credit cache, commitment-bound challenge/response wire types and responder/auditor, responsible-chunk audit, neighbor-sync gossip of commitments, prune-confirmation audit, and the rotation loop. (Originally developed as PR WithAutonomi#113, which this PR replaces; the per-round review history is collapsed here.)
…+ responsible-chunk restore (ADR-0002) Implements ADR-0002 on top of the v12 storage-bound audit base. Two storage audits run side by side: - Storage-commitment audit (NEW, gossip-triggered): on ingesting a peer's commitment, a neighbour pins the just-gossiped root and runs a two-round subtree proof — round 1 rebuilds the nonce-selected contiguous subtree to the pinned root; round 2 demands the original chunk bytes for a nonce-selected sample FROM the audited node (possession is non-delegable; the auditor holds none of the peer's chunks). Pure proof core in subtree.rs; wired auditor/ responder in storage_commitment_audit.rs. Proves a node stores what it CLAIMS (no delete, no relay). - Responsible-chunk audit (RESTORED, periodic): audits a close peer for the chunks it SHOULD hold (responsibility + prior repair hint), via per-key possession digests. Lives in audit.rs; driven by start_audit_loop. Proves a node stores what it SHOULD. Retention is gossip-anchored: the responder stays answerable for the current commitment plus its last-2-gossiped roots for GOSSIP_ANSWERABILITY_TTL after each emission. The pruner consults ResponderCommitmentState::is_held before deleting an out-of-range key (so an in-flight round-2 challenge can't false-positive an honest node), and the commitment rebuild commits only to keys the node is still responsible for, so out-of-range chunks stop being re-committed and age out — bounded reprieve, not a permanent pin. retire_current stops advertising a stale commitment (no responsible keys) while keeping it answerable until its TTL; current_for_gossip snapshots-and-marks atomically to avoid emitting a root that retention no longer covers. Accounting: deterministic failures act on the first occurrence; only timeouts are graced (adaptive grace driven by liveness signals, never deterministic failures). Timeout-driven eviction stays gated off this release (TIMEOUT-EVICTION-DISABLED) pending fleet upgrade; confirmed crypto/possession failures report. Density-aware closeness is observe-only. Tested: lib unit (incl. subtree honest-proof across sizes/nonces, retention TTL + retire-current regressions), two PoC attack suites, and live multi-node e2e (honest pass, data-deleter fail, no honest false-positive, pruner-retention veto over the wire). Both audits independently adversarially reviewed (codex xhigh + Claude) across multiple rounds.
Bind a quote's price to the node's audited storage commitment so a node cannot lie about how much it stores to inflate what it charges. The delivered guarantee is a price ceiling: to be paid above the empty-node baseline a node must surrender a signed commitment that passes binding checks before payment and faces audit after it. Node side: - Forced price: quotes price from the live storage commitment's committed key count and pin that commitment; price is the public formula of the count by exact recomputation (never price inversion). Both quote types (single-node and merkle candidate) carry and sign the binding. - The node ships the signed commitment alongside each quote so any receiver can verify the binding without an extra round trip. - Storers re-run the arithmetic on every quote and cross-check each quote's claimed count against the pinned commitment (resolved from the shipped sidecar, gossip within the answerability TTL, or a capped off-hot-path fetch), emitting portable mismatch evidence on a contradiction. - Monetized commitments enter a deterministic first-audit queue (newest-per-peer, dedup on real audit, retried across the per-peer cooldown) so the latest commitment earning a peer money faces an audit soon; minting fresh pins faster than the cooldown forfeits the older ones' coverage, never the newest's. - A new GetCommitmentByPin request resolves a pin off the hot path, rate-limited and negatively cached. Commitment sidecars are stripped before a receipt is persisted or replicated, so stored proofs do not grow. - The commitment wire type, its pin, verification, and the pricing formula move to ant-protocol so the client and node verify identically; this crate re-exports them. Enforcement ships behind observe-only flags for a hard cutover, per the ADR's rollout. Includes the ADR document and implementation-slices notes. Depends on the evmlib and ant-protocol ADR-0003 releases; will not build standalone until those publish.
There was a problem hiding this comment.
Pull request overview
Implements ADR-0003 commitment-bound quote pricing on the node side by binding quote price to the node’s signed storage commitment (pin + committed key count), shipping commitment sidecars with quotes, and extending replication/audit machinery to ingest commitments, fetch by pin, and audit/credit holders accordingly.
Changes:
- Bind quote pricing/signatures to
(committed_key_count, commitment_pin)and ship signed commitment sidecars with quote responses; strip sidecars before persisting/replicating receipts. - Extend replication protocol/types for commitment gossip, subtree audits (ADR-0002), pin-based commitment fetch, pruning retention veto, and holder-credit gating.
- Add extensive PoC + e2e tests and wire them into CI; update node startup to connect replication commitment state into quoting/verifier paths.
Reviewed changes
Copilot reviewed 33 out of 34 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/poc_bootstrap_stall.rs | Adds PoC regression test documenting bootstrap stall attack behavior. |
| tests/poc_audit_handler_live.rs | Adds live LMDB responder integration tests for subtree audit handler branches. |
| tests/e2e/testnet.rs | Adds retry-on-create for flaky transports; ensures replication engine only starts with identity. |
| tests/e2e/subtree_audit_testnet.rs | Adds end-to-end local testnet tests for subtree audit pass/fail scenarios. |
| tests/e2e/security_attacks.rs | Updates proof fixture to include new commitment_sidecars field. |
| tests/e2e/replication.rs | Updates replication tests for new protocol fields and adds prune retention-veto test. |
| tests/e2e/mod.rs | Registers new subtree-audit testnet module. |
| tests/e2e/merkle_payment.rs | Updates merkle candidate signing to include commitment-bound fields. |
| src/storage/handler.rs | Ships commitment sidecars with quotes; strips sidecars from proofs before replication; adds quote-metric resync + tests. |
| src/replication/types.rs | Adds FailureEvidence::QuoteCommitmentMismatch portable evidence variant. |
| src/replication/recent_provers.rs | Introduces bounded TTL recent-prover cache for holder-credit gating. |
| src/replication/quorum.rs | Adds holder-credit predicate variant to quorum evaluation and tests. |
| src/replication/pruning.rs | Adds commitment-based prune veto + TOCTOU guard; adjusts prune audit timeout semantics. |
| src/replication/protocol.rs | Extends replication wire types: subtree audit messages + commitment fetch by pin; piggybacks commitments in neighbor sync. |
| src/replication/neighbor_sync.rs | Threads optional commitment through sync request/response construction paths. |
| src/replication/config.rs | Adds v12 audit/backpressure constants; bumps replication protocol id to v2; revises audit timeout sizing. |
| src/replication/commitment.rs | Adds node-side Merkle tree + signing utilities; re-exports commitment types/pin/verify from ant-protocol. |
| src/replication/bootstrap.rs | Minor comment update. |
| src/payment/quote.rs | Implements commitment-bound quote pricing + commitment-source abstraction; updates signing bytes for both quote types. |
| src/payment/proof.rs | Re-exports added proof deserializer for single-node proofs. |
| src/payment/pricing.rs | Re-exports pricing formula from ant-protocol as single source of truth. |
| src/payment/metrics.rs | Adds authoritative metric overwrite (set_records) for quoting metrics. |
| src/node.rs | Wires replication commitment state into quote generator + verifier; refactors production rewards-address validation. |
| docs/adr/ADR-0003-implementation-slices.md | Adds implementation slicing notes and rollout/cutover clarifications. |
| docs/adr/ADR-0003-commitment-bound-quote-pricing.md | Adds ADR-0003 specification document. |
| docs/adr/ADR-0002-gossip-triggered-contiguous-subtree-audit.md | Adds ADR-0002 specification document. |
| Cargo.toml | Registers new PoC test binaries for explicit CI invocation. |
| .github/workflows/ci.yml | Runs new PoC suites and live handler tests in CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+321
to
+325
| /// The quote price is driven by `records_stored()`. A monotonic store | ||
| /// counter would let a node delete chunks it was paid to hold yet keep | ||
| /// quoting as if it still held everything. Callers pass the authoritative | ||
| /// count of records the node ACTUALLY HOLDS (from the storage layer) so the | ||
| /// price reflects current holdings, including deletions and pruning. |
Comment on lines
+118
to
+122
| /// Test-only: the record count the quote generator currently prices on. | ||
| /// Used to assert that quote-time resync tracks records actually held. | ||
| #[cfg(test)] | ||
| #[must_use] | ||
| pub(crate) fn priced_records_stored(&self) -> usize { |
Comment on lines
+430
to
+435
| /// The quote price is driven by `QuoteGenerator::records_stored()`. Reading | ||
| /// the live LMDB entry count (an O(1) B-tree page-header read) right before | ||
| /// pricing makes the metric deletion-aware: any chunk removed by | ||
| /// [`LmdbStorage::delete`] or by the replication prune pass is reflected | ||
| /// immediately, with no risk of missing a delete path. | ||
| /// |
Comment on lines
+465
to
+466
| // Price on records ACTUALLY HELD, not a monotonic store counter. | ||
| self.resync_quote_metric(); |
Comment on lines
+543
to
+544
| // Price on records ACTUALLY HELD, not a monotonic store counter. | ||
| self.resync_quote_metric(); |
Comment on lines
+1258
to
+1277
| // Drive a real quote; the priced count must equal records held (10), | ||
| // and the price must equal calculate_price(10) — the externally | ||
| // observable contract. | ||
| assert_eq!(priced_records_after_quote(&protocol), 10); | ||
| let price_full = calculate_price(10); | ||
|
|
||
| // Delete 8 of 10 held chunks. | ||
| for addr in addresses.iter().take(8) { | ||
| assert!(protocol.storage().delete(addr).await.expect("delete")); | ||
| } | ||
| // The next quote must price on 2 records, and the price must be the | ||
| // calculate_price(2) value — strictly different from the 10-record | ||
| // price (price is monotonic non-decreasing in records_stored). | ||
| assert_eq!(priced_records_after_quote(&protocol), 2); | ||
| let price_after = calculate_price(2); | ||
| assert!( | ||
| price_after < price_full, | ||
| "deleting data must lower the observable quote price \ | ||
| (full={price_full:?}, after={price_after:?})" | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ADR-0003: commitment-bound quote pricing — node side. Builds on the ADR-0002 storage-audit work.
Binds a quote's price to the node's audited storage commitment so a node cannot lie about how much it stores to inflate what it charges. The delivered guarantee is a price ceiling: to be paid above the empty-node baseline, a node must surrender a signed commitment that passes binding checks before payment and faces audit after it.
What
GetCommitmentByPinfetch), emitting portable mismatch evidence on a contradiction.Enforcement ships behind observe-only flags for a hard cutover, per the ADR's rollout. The ADR document and implementation-slices notes are included.
Release ordering
Depends on evmlib WithAutonomi/evmlib#11 and ant-protocol WithAutonomi/ant-protocol#15. Built locally via temporary
[patch.crates-io]path patches (not committed here); will not build standalone until those publish.Companion: ant-client resolve-before-pay (the client half).