[BREAKING][Refactor] Simplify get_meta loop logics#158
Conversation
Signed-off-by: 0oshowero0 <[email protected]>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughThe Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login. Comment |
There was a problem hiding this comment.
Pull request overview
This PR refactors the get_metadata method's loop logic by moving retry/timeout handling from scan_data_status to its caller, simplifying the control flow and improving error messages.
Key Changes:
- Removed retry and timeout logic from
scan_data_status, making it a simple delegation function - Changed
get_metadataloop from usingcontinueto usingbreakwithelseclause for clearer control flow - Improved error messages to be more descriptive with task names, partition IDs, and field information
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if len(ready_for_consume_indexes) < batch_size: | ||
| raise RuntimeError( | ||
| "Unexpected error: ready_for_consume_indexes has insufficient samples before sampling. " | ||
| ) | ||
|
|
There was a problem hiding this comment.
This error check is unreachable code. The loop above at line 839 only breaks when len(ready_for_consume_indexes) >= batch_size, so this condition can never be true. This check should be removed.
| if len(ready_for_consume_indexes) < batch_size: | |
| raise RuntimeError( | |
| "Unexpected error: ready_for_consume_indexes has insufficient samples before sampling. " | |
| ) |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
Signed-off-by: 0oshowero0 <[email protected]>
Signed-off-by: 0oshowero0 <[email protected]>
scan_data_status: just return all the samples that meets the requirement. No while loop is needed here.get_metadataloop. We do not need to sample during each loop, just try to scan_data_status.CC @NINGBENZHE
Summary by CodeRabbit
Release Notes
Refactor
Bug Fixes
✏️ Tip: You can customize this high-level summary in your review settings.