Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Wip tentacle lcprefix#67614

Open
mattbenjamin wants to merge 2 commits intoceph:tentaclefrom
linuxbox2:wip-tentacle-lcprefix
Open

Wip tentacle lcprefix#67614
mattbenjamin wants to merge 2 commits intoceph:tentaclefrom
linuxbox2:wip-tentacle-lcprefix

Conversation

@mattbenjamin
Copy link
Contributor

@mattbenjamin mattbenjamin commented Mar 2, 2026

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

kkeshava-akamai and others added 2 commits March 2, 2026 14:12
Signed-off-by: mheler <[email protected]>
(cherry picked from commit 753a55d)
Group lifecycle rules by prefix and process them in one bucket listing,
filtering invalid or zone-incompatible rules before scanning so we avoid
rescanning the bucket for each rule.

rgw/lc: change from spawn-per-rule to spawn-per-object

Currently, bucket_lc_process() spawns one coroutine per rule per
object. This means for a bucket with N lifecycle rules, we spawn
N coroutines for each object.

This creates a workpool utilization problem when there are many
rules. With the default rgw_lc_max_wp_worker=128 and 64 rules,
only ~2 objects can be processed concurrently (128 / 64 ≈ 2).
This severely limits parallelism and causes performance degradation
when multiple tag-based or other rules are configured.

This commit changes the concurrency model to spawn one coroutine
per object instead. Each coroutine processes all applicable rules
for that object sequentially. This allows the full workpool of 128
coroutines to process 128 objects concurrently, regardless of the
number of rules.

Additionally, this fixes a timing issue where rule.update() was
called inside the async coroutine, potentially reading stale data
from the shared LCObjsLister after it had advanced. Now all rules
are updated before spawning to capture the correct lister state.

rgw/lc: reuse prefetched object tags across lifecycle filter

Prefetch tags once per object and pass them into LCOpRule, letting
the tag filter reuse the cached set and skip tag-based rules early
when tags are absent or mismatched.

Signed-off-by: Matthew N. Heler <[email protected]>
Signed-off-by: Matt Benjamin <[email protected]>
(cherry picked from commit 3bb2e52509feb6c219517a9ea73daeeaff68c203)
Signed-off-by: Matt Benjamin <[email protected]>
@mattbenjamin mattbenjamin requested a review from cbodley March 2, 2026 19:14
@mattbenjamin mattbenjamin requested a review from a team as a code owner March 2, 2026 19:14
@github-actions github-actions bot added this to the tentacle milestone Mar 2, 2026
@github-actions
Copy link

github-actions bot commented Mar 2, 2026

Config Diff Tool Output

+ added: rgw_lc_ordered_list_threshold (rgw.yaml.in)
+ added: rgw_lc_list_cnt (rgw.yaml.in)

The above configuration changes are found in the PR. Please update the relevant release documentation if necessary.
Ignore this comment if docs are already updated. To make the "Check ceph config changes" CI check pass, please comment /config check ok and re-run the test.

@mattbenjamin
Copy link
Contributor Author

/config check ok

@mattbenjamin
Copy link
Contributor Author

Check ceph config changes

@cbodley
Copy link
Contributor

cbodley commented Mar 2, 2026

(cherry picked from commit 3bb2e52509feb6c219517a9ea73daeeaff68c203)

i didn't spot any problems with the contents of the backport, but if this is intended to go upstream please cherry-pick from a commit that exists on main and document the Conflicts: in the commit message (https://github.com/ceph/ceph/blob/main/SubmittingPatches-backports.rst#cherry-picking-rules)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants