feat: integrate new storage with distribute cache #533
+11,111
−157
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…& fix client tick issue.
…& fix client tick issue.
feat: add manager function. feat: support basic read/write logic for cache node. feat: add read/write logic.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
The basic functionality of new storage has been developed and is currently planned to be interfaced with distributed caching. However, the existing filehandle mechanism and storage does not support the mechanism of version management, so it is necessary to support the import of the version in the version side, in order to facilitate in the distributed cache or other storage module to verify the version of the current block.
Design
Since the current storage will cache the file attr, adding AtomicU64 directly on the external serial node will not be able to utilize this cache, so add a u64 in the serial file attr for recording the logical version.
In the storage trait, add a read interface with version, and set the version number, but in the write handle there is a fetch block operation may need to get the block from the backend, so in the write interface also maintain a version. truncate is the same.
In addition, write is done by a write back task, so the task holds the version of the block when it sends the write.
TODO
Currently, there is no detailed maintenance for blocks, so changing some blocks may introduce relatively large version changes, as well as the number of ETCD reads and writes, for example, writing a 20M file with a 512kb block may result in 5000 ETCD version updates.