Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Feature] Local and Remote WeightUpdaters #2848

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Mar 13, 2025
Merged

[Feature] Local and Remote WeightUpdaters #2848

merged 8 commits into from
Mar 13, 2025

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Mar 12, 2025 •
edited
Loading

Stack from ghstack (oldest at bottom):

Update
032af80 
[ghstack-poisoned]
This was referenced Mar 11, 2025
Merged
Merged
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 12, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2848

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens pushed a commit that referenced this pull request Mar 12, 2025
[Feature] Local and Remote WeightUpdaters
514ea14 
ghstack-source-id: a15a5a1
Pull Request resolved: #2848
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 12, 2025
Update
6634af4 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 12, 2025
[Feature] Local and Remote WeightUpdaters
9fd9dc5 
ghstack-source-id: 579385c
Pull Request resolved: #2848
@github-actions
Copy link

github-actions bot commented Mar 12, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}18$. Worsened: $\large\color{#d91a1a}17$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.6312s 0.5367s 1.8632 Ops/s 1.8477 Ops/s $\color{#35bf28}+0.84\%$
test_transformed 1.1595s 1.0571s 0.9460 Ops/s 0.9367 Ops/s $\color{#35bf28}+0.99\%$
test_serial 1.5584s 1.5549s 0.6431 Ops/s 0.6289 Ops/s $\color{#35bf28}+2.26\%$
test_parallel 1.4325s 1.3349s 0.7491 Ops/s 0.7460 Ops/s $\color{#35bf28}+0.42\%$
test_step_mdp_speed[True-True-True-True-True] 0.3895ms 30.6548μs 32.6214 KOps/s 32.9746 KOps/s $\color{#d91a1a}-1.07\%$
test_step_mdp_speed[True-True-True-True-False] 57.2070μs 17.9030μs 55.8567 KOps/s 55.9913 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-True-False-True] 66.2230μs 17.3140μs 57.7567 KOps/s 57.6614 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-True-False-False] 49.8530μs 10.1438μs 98.5824 KOps/s 98.0098 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-True-False-True-True] 86.4810μs 32.4619μs 30.8054 KOps/s 30.5467 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[True-True-False-True-False] 74.9700μs 19.7393μs 50.6603 KOps/s 50.2004 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[True-True-False-False-True] 62.9070μs 19.2801μs 51.8670 KOps/s 51.8860 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-False-False-False] 1.5922ms 12.2402μs 81.6978 KOps/s 77.3656 KOps/s $\textbf{\color{#35bf28}+5.60\%}$
test_step_mdp_speed[True-False-True-True-True] 74.3780μs 34.6020μs 28.9001 KOps/s 29.2602 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[True-False-True-True-False] 70.5010μs 21.8636μs 45.7381 KOps/s 46.2090 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-False-True-False-True] 72.5350μs 19.3412μs 51.7030 KOps/s 52.0756 KOps/s $\color{#d91a1a}-0.72\%$
test_step_mdp_speed[True-False-True-False-False] 49.2110μs 12.1682μs 82.1812 KOps/s 83.0579 KOps/s $\color{#d91a1a}-1.06\%$
test_step_mdp_speed[True-False-False-True-True] 86.1000μs 36.4255μs 27.4533 KOps/s 27.7157 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[True-False-False-True-False] 66.4530μs 23.8529μs 41.9237 KOps/s 42.1892 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-False-False-False-True] 60.5230μs 21.1140μs 47.3619 KOps/s 47.3786 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-False-False-False] 50.0130μs 13.8412μs 72.2482 KOps/s 72.5589 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-True-True-True-True] 84.8280μs 34.5042μs 28.9820 KOps/s 28.9519 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-True-False] 59.7010μs 21.9880μs 45.4793 KOps/s 46.0743 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-True-True-False-True] 66.0530μs 22.0768μs 45.2964 KOps/s 45.3232 KOps/s $\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-True-True-False-False] 52.9290μs 13.4940μs 74.1068 KOps/s 74.2789 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-True-False-True-True] 2.4491ms 36.5053μs 27.3933 KOps/s 27.1070 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-True-False-True-False] 75.5410μs 23.6882μs 42.2151 KOps/s 41.8349 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-False-True] 60.9530μs 23.8515μs 41.9262 KOps/s 42.0067 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-True-False-False-False] 62.3460μs 15.2250μs 65.6816 KOps/s 65.3933 KOps/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[False-False-True-True-True] 79.6170μs 38.5017μs 25.9729 KOps/s 26.1083 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-True-True-False] 63.6380μs 25.6308μs 39.0156 KOps/s 38.9525 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[False-False-True-False-True] 76.4120μs 23.7800μs 42.0521 KOps/s 41.5426 KOps/s $\color{#35bf28}+1.23\%$
test_step_mdp_speed[False-False-True-False-False] 51.3650μs 15.3565μs 65.1190 KOps/s 65.4580 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-False-False-True-True] 80.0800μs 40.0753μs 24.9531 KOps/s 23.9277 KOps/s $\color{#35bf28}+4.29\%$
test_step_mdp_speed[False-False-False-True-False] 65.9620μs 27.1581μs 36.8215 KOps/s 37.1951 KOps/s $\color{#d91a1a}-1.00\%$
test_step_mdp_speed[False-False-False-False-True] 68.5980μs 25.4069μs 39.3594 KOps/s 39.5501 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-False-False-False] 68.4170μs 17.3736μs 57.5586 KOps/s 58.7848 KOps/s $\color{#d91a1a}-2.09\%$
test_values[generalized_advantage_estimate-True-True] 10.7604ms 10.3399ms 96.7124 Ops/s 98.7639 Ops/s $\color{#d91a1a}-2.08\%$
test_values[vec_generalized_advantage_estimate-True-True] 29.2934ms 26.3127ms 38.0044 Ops/s 41.4123 Ops/s $\textbf{\color{#d91a1a}-8.23\%}$
test_values[td0_return_estimate-False-False] 0.2356ms 0.2093ms 4.7789 KOps/s 5.3995 KOps/s $\textbf{\color{#d91a1a}-11.49\%}$
test_values[td1_return_estimate-False-False] 25.9240ms 25.6067ms 39.0523 Ops/s 39.4980 Ops/s $\color{#d91a1a}-1.13\%$
test_values[vec_td1_return_estimate-False-False] 31.0803ms 26.6334ms 37.5468 Ops/s 41.2468 Ops/s $\textbf{\color{#d91a1a}-8.97\%}$
test_values[td_lambda_return_estimate-True-False] 40.3619ms 36.6202ms 27.3074 Ops/s 27.5465 Ops/s $\color{#d91a1a}-0.87\%$
test_values[vec_td_lambda_return_estimate-True-False] 28.8744ms 26.4446ms 37.8149 Ops/s 41.1222 Ops/s $\textbf{\color{#d91a1a}-8.04\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9800ms 8.7876ms 113.7964 Ops/s 113.9842 Ops/s $\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.7900ms 1.9859ms 503.5585 Ops/s 500.2860 Ops/s $\color{#35bf28}+0.65\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5622ms 0.3757ms 2.6614 KOps/s 2.6427 KOps/s $\color{#35bf28}+0.71\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.2789ms 42.7790ms 23.3760 Ops/s 23.8899 Ops/s $\color{#d91a1a}-2.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7122ms 3.4739ms 287.8597 Ops/s 277.1545 Ops/s $\color{#35bf28}+3.86\%$
test_dqn_speed[False-None] 6.1268ms 1.4312ms 698.7254 Ops/s 688.1979 Ops/s $\color{#35bf28}+1.53\%$
test_dqn_speed[False-backward] 2.2817ms 1.9330ms 517.3386 Ops/s 514.5777 Ops/s $\color{#35bf28}+0.54\%$
test_dqn_speed[True-None] 0.7512ms 0.5691ms 1.7573 KOps/s 1.7451 KOps/s $\color{#35bf28}+0.70\%$
test_dqn_speed[True-backward] 1.0573ms 0.9892ms 1.0109 KOps/s 998.2078 Ops/s $\color{#35bf28}+1.27\%$
test_dqn_speed[reduce-overhead-None] 0.8497ms 0.5692ms 1.7567 KOps/s 1.7537 KOps/s $\color{#35bf28}+0.17\%$
test_dqn_speed[reduce-overhead-backward] 1.0612ms 0.9888ms 1.0113 KOps/s 1.0061 KOps/s $\color{#35bf28}+0.52\%$
test_ddpg_speed[False-None] 3.3398ms 2.9410ms 340.0243 Ops/s 339.3237 Ops/s $\color{#35bf28}+0.21\%$
test_ddpg_speed[False-backward] 4.4174ms 4.1483ms 241.0618 Ops/s 244.7073 Ops/s $\color{#d91a1a}-1.49\%$
test_ddpg_speed[True-None] 2.1056ms 1.4598ms 685.0039 Ops/s 686.2088 Ops/s $\color{#d91a1a}-0.18\%$
test_ddpg_speed[True-backward] 2.5157ms 2.3972ms 417.1521 Ops/s 418.1893 Ops/s $\color{#d91a1a}-0.25\%$
test_ddpg_speed[reduce-overhead-None] 2.0669ms 1.4659ms 682.1753 Ops/s 683.1094 Ops/s $\color{#d91a1a}-0.14\%$
test_ddpg_speed[reduce-overhead-backward] 2.5396ms 2.3975ms 417.1081 Ops/s 414.8056 Ops/s $\color{#35bf28}+0.56\%$
test_sac_speed[False-None] 9.7689ms 8.4471ms 118.3845 Ops/s 121.7069 Ops/s $\color{#d91a1a}-2.73\%$
test_sac_speed[False-backward] 11.9043ms 11.2489ms 88.8979 Ops/s 91.0541 Ops/s $\color{#d91a1a}-2.37\%$
test_sac_speed[True-None] 3.1066ms 2.5963ms 385.1625 Ops/s 372.9457 Ops/s $\color{#35bf28}+3.28\%$
test_sac_speed[True-backward] 5.1333ms 4.5481ms 219.8702 Ops/s 234.5119 Ops/s $\textbf{\color{#d91a1a}-6.24\%}$
test_sac_speed[reduce-overhead-None] 3.8836ms 2.7827ms 359.3571 Ops/s 379.7662 Ops/s $\textbf{\color{#d91a1a}-5.37\%}$
test_sac_speed[reduce-overhead-backward] 5.6527ms 4.7772ms 209.3286 Ops/s 231.1188 Ops/s $\textbf{\color{#d91a1a}-9.43\%}$
test_redq_speed[False-None] 18.8256ms 13.8607ms 72.1466 Ops/s 76.7897 Ops/s $\textbf{\color{#d91a1a}-6.05\%}$
test_redq_speed[False-backward] 24.1138ms 22.9117ms 43.6458 Ops/s 44.5605 Ops/s $\color{#d91a1a}-2.05\%$
test_redq_speed[True-None] 8.4165ms 7.5609ms 132.2596 Ops/s 150.9342 Ops/s $\textbf{\color{#d91a1a}-12.37\%}$
test_redq_speed[True-backward] 16.5836ms 15.3420ms 65.1804 Ops/s 67.6069 Ops/s $\color{#d91a1a}-3.59\%$
test_redq_speed[reduce-overhead-None] 8.8205ms 7.6573ms 130.5944 Ops/s 140.3795 Ops/s $\textbf{\color{#d91a1a}-6.97\%}$
test_redq_speed[reduce-overhead-backward] 15.6849ms 15.3114ms 65.3109 Ops/s 66.0143 Ops/s $\color{#d91a1a}-1.07\%$
test_redq_deprec_speed[False-None] 16.6270ms 13.8454ms 72.2259 Ops/s 71.9641 Ops/s $\color{#35bf28}+0.36\%$
test_redq_deprec_speed[False-backward] 21.8809ms 19.8973ms 50.2581 Ops/s 51.6881 Ops/s $\color{#d91a1a}-2.77\%$
test_redq_deprec_speed[True-None] 6.4019ms 5.6099ms 178.2564 Ops/s 187.8116 Ops/s $\textbf{\color{#d91a1a}-5.09\%}$
test_redq_deprec_speed[True-backward] 11.2735ms 11.0346ms 90.6243 Ops/s 96.8098 Ops/s $\textbf{\color{#d91a1a}-6.39\%}$
test_redq_deprec_speed[reduce-overhead-None] 7.3742ms 5.8936ms 169.6760 Ops/s 189.3440 Ops/s $\textbf{\color{#d91a1a}-10.39\%}$
test_redq_deprec_speed[reduce-overhead-backward] 11.7272ms 11.0641ms 90.3825 Ops/s 95.3060 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_td3_speed[False-None] 8.8771ms 8.4335ms 118.5751 Ops/s 118.9952 Ops/s $\color{#d91a1a}-0.35\%$
test_td3_speed[False-backward] 11.7155ms 11.0081ms 90.8425 Ops/s 88.6622 Ops/s $\color{#35bf28}+2.46\%$
test_td3_speed[True-None] 2.5105ms 2.3441ms 426.6043 Ops/s 434.7790 Ops/s $\color{#d91a1a}-1.88\%$
test_td3_speed[True-backward] 4.5712ms 4.0993ms 243.9463 Ops/s 244.0833 Ops/s $\color{#d91a1a}-0.06\%$
test_td3_speed[reduce-overhead-None] 3.2374ms 2.4256ms 412.2633 Ops/s 432.6428 Ops/s $\color{#d91a1a}-4.71\%$
test_td3_speed[reduce-overhead-backward] 4.9367ms 4.1832ms 239.0502 Ops/s 252.7312 Ops/s $\textbf{\color{#d91a1a}-5.41\%}$
test_cql_speed[False-None] 40.4301ms 37.5380ms 26.6397 Ops/s 27.0169 Ops/s $\color{#d91a1a}-1.40\%$
test_cql_speed[False-backward] 52.3087ms 49.1623ms 20.3408 Ops/s 20.6469 Ops/s $\color{#d91a1a}-1.48\%$
test_cql_speed[True-None] 25.0895ms 22.9112ms 43.6469 Ops/s 44.4439 Ops/s $\color{#d91a1a}-1.79\%$
test_cql_speed[True-backward] 46.1538ms 30.4475ms 32.8435 Ops/s 33.9668 Ops/s $\color{#d91a1a}-3.31\%$
test_cql_speed[reduce-overhead-None] 23.3970ms 22.4186ms 44.6057 Ops/s 42.7296 Ops/s $\color{#35bf28}+4.39\%$
test_cql_speed[reduce-overhead-backward] 31.4456ms 29.0478ms 34.4260 Ops/s 34.3669 Ops/s $\color{#35bf28}+0.17\%$
test_a2c_speed[False-None] 10.3493ms 7.3524ms 136.0107 Ops/s 138.3185 Ops/s $\color{#d91a1a}-1.67\%$
test_a2c_speed[False-backward] 16.4746ms 14.2941ms 69.9589 Ops/s 70.8782 Ops/s $\color{#d91a1a}-1.30\%$
test_a2c_speed[True-None] 5.1851ms 4.6761ms 213.8545 Ops/s 215.2704 Ops/s $\color{#d91a1a}-0.66\%$
test_a2c_speed[True-backward] 12.5267ms 11.1432ms 89.7411 Ops/s 85.4254 Ops/s $\textbf{\color{#35bf28}+5.05\%}$
test_a2c_speed[reduce-overhead-None] 5.3692ms 4.6638ms 214.4154 Ops/s 209.0047 Ops/s $\color{#35bf28}+2.59\%$
test_a2c_speed[reduce-overhead-backward] 12.7091ms 11.0881ms 90.1871 Ops/s 88.4763 Ops/s $\color{#35bf28}+1.93\%$
test_ppo_speed[False-None] 9.0832ms 7.5141ms 133.0835 Ops/s 132.0537 Ops/s $\color{#35bf28}+0.78\%$
test_ppo_speed[False-backward] 16.3092ms 14.6963ms 68.0441 Ops/s 67.7265 Ops/s $\color{#35bf28}+0.47\%$
test_ppo_speed[True-None] 5.4759ms 5.0850ms 196.6567 Ops/s 196.2167 Ops/s $\color{#35bf28}+0.22\%$
test_ppo_speed[True-backward] 11.9285ms 10.9753ms 91.1134 Ops/s 90.8324 Ops/s $\color{#35bf28}+0.31\%$
test_ppo_speed[reduce-overhead-None] 6.0056ms 5.1016ms 196.0180 Ops/s 197.5300 Ops/s $\color{#d91a1a}-0.77\%$
test_ppo_speed[reduce-overhead-backward] 11.6858ms 10.9431ms 91.3819 Ops/s 89.0530 Ops/s $\color{#35bf28}+2.62\%$
test_reinforce_speed[False-None] 8.4796ms 6.6572ms 150.2129 Ops/s 147.0465 Ops/s $\color{#35bf28}+2.15\%$
test_reinforce_speed[False-backward] 11.9584ms 10.3385ms 96.7256 Ops/s 24.1167 Ops/s $\textbf{\color{#35bf28}+301.07\%}$
test_reinforce_speed[True-None] 4.8733ms 4.0683ms 245.8044 Ops/s 231.6346 Ops/s $\textbf{\color{#35bf28}+6.12\%}$
test_reinforce_speed[True-backward] 10.8772ms 9.9670ms 100.3307 Ops/s 91.9923 Ops/s $\textbf{\color{#35bf28}+9.06\%}$
test_reinforce_speed[reduce-overhead-None] 5.6914ms 4.1031ms 243.7207 Ops/s 234.0356 Ops/s $\color{#35bf28}+4.14\%$
test_reinforce_speed[reduce-overhead-backward] 10.2624ms 9.9093ms 100.9156 Ops/s 98.0169 Ops/s $\color{#35bf28}+2.96\%$
test_iql_speed[False-None] 39.8749ms 33.2200ms 30.1023 Ops/s 29.5764 Ops/s $\color{#35bf28}+1.78\%$
test_iql_speed[False-backward] 48.8941ms 45.2258ms 22.1113 Ops/s 21.4801 Ops/s $\color{#35bf28}+2.94\%$
test_iql_speed[True-None] 17.3903ms 15.5805ms 64.1829 Ops/s 63.3094 Ops/s $\color{#35bf28}+1.38\%$
test_iql_speed[True-backward] 28.4500ms 27.0669ms 36.9455 Ops/s 36.4025 Ops/s $\color{#35bf28}+1.49\%$
test_iql_speed[reduce-overhead-None] 27.7810ms 16.2117ms 61.6837 Ops/s 62.3224 Ops/s $\color{#d91a1a}-1.02\%$
test_iql_speed[reduce-overhead-backward] 29.2619ms 26.7796ms 37.3419 Ops/s 35.9429 Ops/s $\color{#35bf28}+3.89\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2239ms 4.7396ms 210.9894 Ops/s 197.6558 Ops/s $\textbf{\color{#35bf28}+6.75\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2475ms 0.5163ms 1.9368 KOps/s 1.8248 KOps/s $\textbf{\color{#35bf28}+6.14\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7837ms 0.4931ms 2.0280 KOps/s 1.9486 KOps/s $\color{#35bf28}+4.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.2919ms 4.4926ms 222.5883 Ops/s 201.9031 Ops/s $\textbf{\color{#35bf28}+10.25\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0033ms 0.5088ms 1.9655 KOps/s 1.9277 KOps/s $\color{#35bf28}+1.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7207ms 0.4777ms 2.0935 KOps/s 2.0176 KOps/s $\color{#35bf28}+3.76\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5465ms 1.6769ms 596.3271 Ops/s 601.3851 Ops/s $\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3290ms 1.5824ms 631.9522 Ops/s 631.5393 Ops/s $\color{#35bf28}+0.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.4744ms 4.6987ms 212.8252 Ops/s 203.5311 Ops/s $\color{#35bf28}+4.57\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.7384ms 0.6762ms 1.4788 KOps/s 1.5012 KOps/s $\color{#d91a1a}-1.49\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9871ms 0.6326ms 1.5807 KOps/s 1.5564 KOps/s $\color{#35bf28}+1.56\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3477ms 4.5916ms 217.7913 Ops/s 207.7402 Ops/s $\color{#35bf28}+4.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.1652ms 0.5124ms 1.9517 KOps/s 1.8818 KOps/s $\color{#35bf28}+3.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7546ms 0.4880ms 2.0492 KOps/s 1.9478 KOps/s $\textbf{\color{#35bf28}+5.21\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7591ms 4.4380ms 225.3244 Ops/s 210.5606 Ops/s $\textbf{\color{#35bf28}+7.01\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.1831ms 0.5136ms 1.9470 KOps/s 1.9346 KOps/s $\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7924ms 0.4816ms 2.0766 KOps/s 2.0439 KOps/s $\color{#35bf28}+1.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.9021ms 4.6601ms 214.5889 Ops/s 200.3163 Ops/s $\textbf{\color{#35bf28}+7.13\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.2164ms 0.6538ms 1.5295 KOps/s 1.4664 KOps/s $\color{#35bf28}+4.30\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0148ms 0.6386ms 1.5659 KOps/s 1.5640 KOps/s $\color{#35bf28}+0.12\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.7545s 19.2980ms 51.8187 Ops/s 239.5266 Ops/s $\textbf{\color{#d91a1a}-78.37\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.1193ms 2.3812ms 419.9550 Ops/s 370.3866 Ops/s $\textbf{\color{#35bf28}+13.38\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.9909ms 1.3126ms 761.8371 Ops/s 728.3980 Ops/s $\color{#35bf28}+4.59\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.2522ms 4.3091ms 232.0663 Ops/s 240.6867 Ops/s $\color{#d91a1a}-3.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 5.1292ms 2.3087ms 433.1485 Ops/s 411.2969 Ops/s $\textbf{\color{#35bf28}+5.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 8.4520ms 1.5815ms 632.3255 Ops/s 743.3152 Ops/s $\textbf{\color{#d91a1a}-14.93\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.4247ms 4.4716ms 223.6332 Ops/s 22.7629 Ops/s $\textbf{\color{#35bf28}+882.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.3082ms 2.4874ms 402.0244 Ops/s 367.0008 Ops/s $\textbf{\color{#35bf28}+9.54\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.6283ms 1.5488ms 645.6472 Ops/s 546.2660 Ops/s $\textbf{\color{#35bf28}+18.19\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 64.4067ms 49.8773ms 20.0492 Ops/s 19.7606 Ops/s $\color{#35bf28}+1.46\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 25.7715ms 15.0343ms 66.5145 Ops/s 67.3962 Ops/s $\color{#d91a1a}-1.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 63.4524ms 49.2152ms 20.3189 Ops/s 18.9469 Ops/s $\textbf{\color{#35bf28}+7.24\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 16.6235ms 14.5748ms 68.6117 Ops/s 64.5800 Ops/s $\textbf{\color{#35bf28}+6.24\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 65.4432ms 51.0497ms 19.5888 Ops/s 19.3419 Ops/s $\color{#35bf28}+1.28\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 18.2841ms 15.8186ms 63.2166 Ops/s 61.5525 Ops/s $\color{#35bf28}+2.70\%$

Update
677e543 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 12, 2025
[Feature] Local and Remote WeightUpdaters
ed68115 
ghstack-source-id: d361b5a
Pull Request resolved: #2848
@vmoens vmoens added the enhancement New feature or request label Mar 13, 2025
Update
ba43867 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
d5aa430 
ghstack-source-id: 0aafc7e
Pull Request resolved: #2848
Update
e748814 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
54bf9a6 
ghstack-source-id: 2618b89
Pull Request resolved: #2848
Update
93ba882 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
806ebde 
ghstack-source-id: b070441
Pull Request resolved: #2848
Update
19f2e67 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
0591ff3 
ghstack-source-id: c69723c
Pull Request resolved: #2848
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
7904d7d 
ghstack-source-id: c69723c
Pull Request resolved: #2848
Update
a19a8fe 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
b902255 
ghstack-source-id: 2962530
Pull Request resolved: #2848
@vmoens vmoens merged commit a19a8fe into gh/vmoens/115/base Mar 13, 2025
15 of 20 checks passed
vmoens pushed a commit that referenced this pull request Mar 13, 2025
[Feature] Local and Remote WeightUpdaters
27d3680 
ghstack-source-id: 2962530
Pull Request resolved: #2848
@vmoens vmoens deleted the gh/vmoens/115/head branch March 13, 2025 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens @facebook-github-bot