-
Notifications
You must be signed in to change notification settings - Fork 242
Add progress handling during hash index generation for adaptive mode 'block-hash-index' #1734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This allows installing the adaptive-enabled bundle test/good-adaptive-meta-bundle.raucb from the emulated system. Signed-off-by: Enrico Jörns <[email protected]>
The name TEST_UPDATE_HANDLER_INCR_BLOCK_HASH_IDX is a remnant from the initial term 'incremental updates' that is now referred to as 'adaptive updates'. Rename it to allow finding it when searching for 'adaptive'. Signed-off-by: Enrico Jörns <[email protected]>
… adaptive mode Depending on the CPU and storage speed, generating the hash index for the adaptive mode 'block-hash-index' might take a notable amount of time. So far, we don't emit any progress information during these steps, which could give the impression that the installation progress stalls. With this simple approach, we reserve 10% of the progress for both the 'target_slot' and the 'active_slot' generation. While this does not emit progress *during* the actual hash index generation, it adds two benefits: 1. An update between the two hash indexes that might need to be generated 2. A progress of already 20% after reading or generating the hash index. Which might well reflect the optimization even if the index was loaded from a file. Since 10% is a quite arbitrary value, define it by R_HASH_INDEX_GEN_PROGRESS_SPAN to allow changing it easily and to reflect its meaning (also for follow-up code added). Signed-off-by: Enrico Jörns <[email protected]>
The newly introduced method r_context_inc_step_percentage() allows incrementing the given step's percentage by one, without needing to now its current absolute value. Signed-off-by: Enrico Jörns <[email protected]>
While the previous change added progress updates after each possible hash index generation, this now introduces progress updates *during* hash generation. We have reserved a fixed amount of step percentage for each generation, represented by the R_HASH_INDEX_GEN_PROGRESS_SPAN variable. By splitting up the generation into R_HASH_INDEX_GEN_PROGRESS_SPAN - 1 steps and increasing the step percentage by one each time, we ensure emitting a linear progress for each generation. Signed-off-by: Enrico Jörns <[email protected]>
With the previous addition of progress handling for adaptive mode hash index generation, the progress may already be at a certain percentage when an adaptive error occurs that triggers the manual copying fallback. The manual copying will then reset the step progress to 0. Since we allow the step progress to go backwards but ensure that high level progress is only emitted when progress increases, this shouldn't be an issue. The only observable behavior in this case is that the overall progress may stall until the step percentage exceeds its previous value again. Since this case should happen rarely, it should be acceptable. Signed-off-by: Enrico Jörns <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1734 +/- ##
==========================================
+ Coverage 84.51% 84.55% +0.04%
==========================================
Files 76 76
Lines 22328 22412 +84
==========================================
+ Hits 18871 18951 +80
- Misses 3457 3461 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
While this solution is more like an intermediate workaround, I currently don't have more time in my project for a larger rework of the current progress handling, which would be required to allow more dynamic sub-step creation, etc. Maybe I'll come back to this in some other context later, but for now, this is the best solution I can provide. If someone is interested in improving progress updates during block-hash-index generation, please drop a note here. |
Signed-off-by: Enrico Jörns <[email protected]>
206a149 to
80a8852
Compare
|
I've force pushed (https://github.com/rauc/rauc/compare/a87e8bd17478..80a8852a573d) to fix the test case when running it separately. The |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like working around the nested step structure by calculating ranges explicitly, but it's better than not having progress while generating the index. We'll have to refactor this later.
Depending on the CPU and storage speed, generating the hash index for the adaptive mode 'block-hash-index' might take a notable amount of time.
So far, we don't emit any progress information during these steps, which could give the impression that the installation progress stalls.
With this approach, we reserve 10% of the step progress for both the 'target_slot' and the 'active_slot' generation and increment the progress evenly while generating the hash index.
Also, add some testing adaptations and improvements related to this feature.