[adb] [sync] Re-use existing persisted data during state sync #1275
Conversation
| .await | ||
| .unwrap(); | ||
|
|
||
| // Verify database state |
There was a problem hiding this comment.
Moved from below to be more consistent with other tests
There was a problem hiding this comment.
Pull Request Overview
This PR implements the ability to reuse existing persisted data during state synchronization in ADB, addressing issue #1213. The changes enable the sync process to detect and leverage previously stored operations and MMR data that fall within the sync boundaries, rather than always starting from scratch.
Key changes include:
- Modified
init_syncfunctions in both MMR and Journal to handle three scenarios: fresh start, prune and reuse, and prune and rewind - Updated sync client to detect when existing data matches sync targets and skip network operations
- Enhanced configuration structures to use boundary-based parameters instead of single pruning positions
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| storage/src/mmr/mod.rs | Added InvalidSize error variant for MMR validation |
| storage/src/mmr/journaled.rs | Replaced init_pruned with init_sync to handle data reuse scenarios and updated configuration |
| storage/src/journal/fixed.rs | Replaced init_pruned with init_sync to support boundary-based initialization and data reuse |
| storage/src/adb/any/sync/resolver.rs | Added test helper FailResolver for testing scenarios where no network operations should occur |
| storage/src/adb/any/sync/mod.rs | Modified sync function to handle cases where sync is already complete |
| storage/src/adb/any/sync/client.rs | Enhanced client to detect complete sync scenarios and avoid unnecessary network operations |
| storage/src/adb/any/mod.rs | Updated init_synced to handle existing data scenarios and renamed configuration parameters |
| } | ||
|
|
||
| // Install pinned nodes on first successful batch. | ||
| if log_size == config.lower_bound_ops { |
There was a problem hiding this comment.
I'm assuming this was the "nuance" you referred to lol?
There was a problem hiding this comment.
I was more referring to this:
if cfg.lower_bound < journal_size {
// We need to add the pinned nodes required for proving at the given pruning boundary.
let mut additional_pinned_nodes = HashMap::new();
for pos in Proof::<H::Digest>::nodes_to_pin(cfg.lower_bound) {
let digest =
Mmr::<E, H>::get_from_metadata_or_journal(&metadata, &journal, pos).await?;
additional_pinned_nodes.insert(pos, digest);
}
mem_mmr.add_pinned_nodes(additional_pinned_nodes);
}…n jixed journal init_sync
Codecov ReportAttention: Patch coverage is
@@ Coverage Diff @@
## main #1275 +/- ##
==========================================
+ Coverage 91.06% 91.09% +0.03%
==========================================
Files 246 246
Lines 60120 60489 +369
==========================================
+ Hits 54748 55103 +355
- Misses 5372 5386 +14
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This PR addresses #1213 by allowing for the re-use of existing persisted data when entering state sync.