[sync] Factor out sync engine; implement sync for adb::Immutable
#1341
Conversation
storage/src/journal/variable.rs
Outdated
| /// - No section index < `lower_bound / items_per_section` exists. | ||
| /// - No section index > `upper_bound / items_per_section` exists. | ||
| /// - The last retained section is truncated so that its last item’s location is `<= upper_bound`. | ||
| pub(crate) async fn init_sync( |
There was a problem hiding this comment.
I know we did this already with fixed journal, but wondering if this sync-client-specific code should be outside the core journal module. It seems more application specific?
There was a problem hiding this comment.
Fair point.
I moved the fixed journal's init_sync and helpers to adb/any/fixed/sync.rs and made them functions rather than methods.
For the variable journal, I moved them to adb/any/variable/sync.rs. I'm not sure this is the right place because:
- Sync isn't yet implemented for
any::variable::Anyyet - These functions are used by
any::Immutable
but I'm not immediately sure where a better place to put this code would be.
More generally, I'd like to hear others' thoughts on how we should organize the sync-related code. As it stands, there are sync modules for any::fixed, any::variable, any::immutable. An alternative would be to consolidate all sync-related implementations for these types in one place/module.
There was a problem hiding this comment.
Or perhaps I misunderstood your original comment, @roberto-bayardo, and you had in mind a sync module at storage/src/journal/variable/sync.rs?
There was a problem hiding this comment.
Feels more like it belongs in adb somewhere since that's what we're syncing, but if a "syncable journal" is generic enough to be independent of adb, then I guess it could go in journal/ ?
| use crate::adb::sync::engine::Config; | ||
| use commonware_codec::Encode; | ||
|
|
There was a problem hiding this comment.
nit: we don't do this \n elsewhere in the codebase but I do find it clearer 🤔
There was a problem hiding this comment.
Your call. As you say, it's somewhat easier to read at the cost of being inconsistent with mod.rs files.
| /// - Reuse any on-disk data whose logical locations lie within [lower_bound, upper_bound]. | ||
| /// - Discard/ignore any data strictly below `lower_bound` and strictly above `upper_bound`. | ||
| /// - Report `size()` equal to the next location to be filled. | ||
| fn create_journal( |
There was a problem hiding this comment.
nit: I wonder if we should just do create and resize?
I don't think the caller here needs to know/care if it is journal (and/or that is implicit from the types)?
There was a problem hiding this comment.
I think the _journal suffix makes sense. Otherwise, when you look at the pub trait Database and see create, you might assume that it creates a database when actually it creates a journal. Of course, the comment and return type tell you that it's not. But still, I think the extra verbosity is warranted in this case to avoid:
let journal = DB::create() // Feels like this should return a db?There was a problem hiding this comment.
I'm not sure how this database/journal trait will evolve and I'm ok to keep as-is for now (until the other adbs are supported).
Codecov Report❌ Patch coverage is @@ Coverage Diff @@
## main #1341 +/- ##
==========================================
+ Coverage 91.56% 91.69% +0.13%
==========================================
Files 266 275 +9
Lines 67545 68964 +1419
==========================================
+ Hits 61846 63239 +1393
- Misses 5699 5725 +26
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This PR:
Anysync client and its associated types intoadb::sync, and adapts the types to be generic over aSyncabledatabase trait.Syncableforadb::Immutable.sync/exampleto be able to run with either afixed::Anyoradb::Immutabledatabase.sync/examplearchitecture and adds structure to the crate.