Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Refactor] LLM data structures #2834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Mar 6, 2025
Merged

[Refactor] LLM data structures #2834

merged 4 commits into from
Mar 6, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Mar 5, 2025 •
edited
Loading

Update
7fb2a84 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Mar 5, 2025
Merged
@pytorch-bot
Copy link

pytorch-bot bot commented Mar 5, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2834

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens pushed a commit that referenced this pull request Mar 5, 2025
[Refactor] LLM data structures
5af48dd 
ghstack-source-id: 5ae1179
Pull Request resolved: #2834
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 5, 2025
@github-actions
Copy link

github-actions bot commented Mar 5, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.6220s 0.5338s 1.8735 Ops/s 1.9432 Ops/s $\color{#d91a1a}-3.59\%$
test_transformed 1.1413s 1.0477s 0.9545 Ops/s 1.0102 Ops/s $\textbf{\color{#d91a1a}-5.52\%}$
test_serial 1.6416s 1.5345s 0.6517 Ops/s 0.6705 Ops/s $\color{#d91a1a}-2.80\%$
test_parallel 1.4399s 1.3511s 0.7401 Ops/s 0.7610 Ops/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[True-True-True-True-True] 0.1325ms 29.8676μs 33.4810 KOps/s 33.1810 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[True-True-True-True-False] 41.8580μs 17.5748μs 56.8997 KOps/s 56.3600 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-True-True-False-True] 0.7109ms 17.2619μs 57.9309 KOps/s 58.2625 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[True-True-True-False-False] 37.8010μs 10.0114μs 99.8857 KOps/s 98.8926 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-True-False-True-True] 80.5010μs 31.9381μs 31.3106 KOps/s 31.1279 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-True-False-True-False] 50.2830μs 19.5659μs 51.1092 KOps/s 50.5067 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-True-False-False-True] 48.8910μs 18.7976μs 53.1982 KOps/s 52.1229 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[True-True-False-False-False] 44.0020μs 11.8256μs 84.5621 KOps/s 83.5039 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-False-True-True-True] 64.8210μs 33.5211μs 29.8320 KOps/s 29.1594 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-False-True-True-False] 46.5570μs 21.3188μs 46.9069 KOps/s 46.1424 KOps/s $\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-True-False-True] 53.6510μs 18.7099μs 53.4477 KOps/s 52.1001 KOps/s $\color{#35bf28}+2.59\%$
test_step_mdp_speed[True-False-True-False-False] 37.2090μs 11.8354μs 84.4921 KOps/s 84.2328 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-True-True] 79.4780μs 35.4214μs 28.2315 KOps/s 27.9978 KOps/s $\color{#35bf28}+0.83\%$
test_step_mdp_speed[True-False-False-True-False] 46.8770μs 23.0548μs 43.3748 KOps/s 42.8895 KOps/s $\color{#35bf28}+1.13\%$
test_step_mdp_speed[True-False-False-False-True] 49.8230μs 20.4312μs 48.9447 KOps/s 47.8128 KOps/s $\color{#35bf28}+2.37\%$
test_step_mdp_speed[True-False-False-False-False] 39.4330μs 13.5002μs 74.0727 KOps/s 73.2413 KOps/s $\color{#35bf28}+1.14\%$
test_step_mdp_speed[False-True-True-True-True] 96.7630μs 33.5991μs 29.7627 KOps/s 29.1354 KOps/s $\color{#35bf28}+2.15\%$
test_step_mdp_speed[False-True-True-True-False] 78.1130μs 21.5262μs 46.4549 KOps/s 46.3595 KOps/s $\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-True-True-False-True] 49.9730μs 21.6374μs 46.2163 KOps/s 46.0839 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-True-False-False] 49.6330μs 13.2444μs 75.5034 KOps/s 75.0785 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[False-True-False-True-True] 0.6633ms 35.2139μs 28.3979 KOps/s 27.9429 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-True-False-True-False] 2.6649ms 23.1796μs 43.1413 KOps/s 42.8339 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-True-False-False-True] 61.8460μs 23.2914μs 42.9344 KOps/s 42.9149 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[False-True-False-False-False] 56.4560μs 14.9153μs 67.0454 KOps/s 66.4094 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-False-True-True-True] 80.7510μs 37.0827μs 26.9667 KOps/s 26.6441 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-True-True-False] 64.0000μs 25.0060μs 39.9904 KOps/s 39.7046 KOps/s $\color{#35bf28}+0.72\%$
test_step_mdp_speed[False-False-True-False-True] 86.5240μs 23.2978μs 42.9225 KOps/s 42.5623 KOps/s $\color{#35bf28}+0.85\%$
test_step_mdp_speed[False-False-True-False-False] 43.1910μs 14.8801μs 67.2037 KOps/s 66.6013 KOps/s $\color{#35bf28}+0.90\%$
test_step_mdp_speed[False-False-False-True-True] 81.9930μs 38.6048μs 25.9035 KOps/s 25.6319 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-False-True-False] 72.8460μs 26.5656μs 37.6427 KOps/s 37.5940 KOps/s $\color{#35bf28}+0.13\%$
test_step_mdp_speed[False-False-False-False-True] 59.3510μs 24.4976μs 40.8203 KOps/s 39.9409 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-False-False-False-False] 45.0640μs 16.5321μs 60.4883 KOps/s 60.2807 KOps/s $\color{#35bf28}+0.34\%$
test_values[generalized_advantage_estimate-True-True] 10.8710ms 9.7891ms 102.1541 Ops/s 99.1123 Ops/s $\color{#35bf28}+3.07\%$
test_values[vec_generalized_advantage_estimate-True-True] 28.4928ms 26.1450ms 38.2482 Ops/s 41.2536 Ops/s $\textbf{\color{#d91a1a}-7.29\%}$
test_values[td0_return_estimate-False-False] 1.4741ms 0.2049ms 4.8799 KOps/s 5.6190 KOps/s $\textbf{\color{#d91a1a}-13.15\%}$
test_values[td1_return_estimate-False-False] 27.6785ms 24.3991ms 40.9851 Ops/s 41.7292 Ops/s $\color{#d91a1a}-1.78\%$
test_values[vec_td1_return_estimate-False-False] 29.6922ms 26.5526ms 37.6611 Ops/s 41.1937 Ops/s $\textbf{\color{#d91a1a}-8.58\%}$
test_values[td_lambda_return_estimate-True-False] 38.4151ms 35.2744ms 28.3492 Ops/s 28.7377 Ops/s $\color{#d91a1a}-1.35\%$
test_values[vec_td_lambda_return_estimate-True-False] 31.9025ms 26.7391ms 37.3984 Ops/s 40.9508 Ops/s $\textbf{\color{#d91a1a}-8.67\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.0809ms 8.6317ms 115.8524 Ops/s 117.8816 Ops/s $\color{#d91a1a}-1.72\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2097ms 1.8322ms 545.7841 Ops/s 520.2068 Ops/s $\color{#35bf28}+4.92\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5079ms 0.3685ms 2.7141 KOps/s 2.7405 KOps/s $\color{#d91a1a}-0.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.6499ms 42.8058ms 23.3613 Ops/s 22.7164 Ops/s $\color{#35bf28}+2.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.2721ms 3.4399ms 290.7068 Ops/s 290.2568 Ops/s $\color{#35bf28}+0.16\%$
test_dqn_speed[False-None] 5.7665ms 1.4148ms 706.8268 Ops/s 719.5737 Ops/s $\color{#d91a1a}-1.77\%$
test_dqn_speed[False-backward] 2.0787ms 1.9074ms 524.2635 Ops/s 532.7222 Ops/s $\color{#d91a1a}-1.59\%$
test_dqn_speed[True-None] 0.7697ms 0.5546ms 1.8032 KOps/s 1.7750 KOps/s $\color{#35bf28}+1.59\%$
test_dqn_speed[True-backward] 1.0088ms 0.9669ms 1.0342 KOps/s 825.7203 Ops/s $\textbf{\color{#35bf28}+25.25\%}$
test_dqn_speed[reduce-overhead-None] 0.8987ms 0.5625ms 1.7778 KOps/s 1.7809 KOps/s $\color{#d91a1a}-0.18\%$
test_dqn_speed[reduce-overhead-backward] 1.0567ms 0.9728ms 1.0280 KOps/s 1.0202 KOps/s $\color{#35bf28}+0.76\%$
test_ddpg_speed[False-None] 3.6557ms 2.8758ms 347.7338 Ops/s 351.7241 Ops/s $\color{#d91a1a}-1.13\%$
test_ddpg_speed[False-backward] 5.1837ms 4.0704ms 245.6764 Ops/s 252.1285 Ops/s $\color{#d91a1a}-2.56\%$
test_ddpg_speed[True-None] 3.7443ms 1.4323ms 698.1778 Ops/s 696.5557 Ops/s $\color{#35bf28}+0.23\%$
test_ddpg_speed[True-backward] 2.3771ms 2.3160ms 431.7806 Ops/s 428.4979 Ops/s $\color{#35bf28}+0.77\%$
test_ddpg_speed[reduce-overhead-None] 1.9398ms 1.4181ms 705.1518 Ops/s 535.5529 Ops/s $\textbf{\color{#35bf28}+31.67\%}$
test_ddpg_speed[reduce-overhead-backward] 2.3700ms 2.3018ms 434.4356 Ops/s 426.6481 Ops/s $\color{#35bf28}+1.83\%$
test_sac_speed[False-None] 8.5299ms 7.9558ms 125.6942 Ops/s 126.3590 Ops/s $\color{#d91a1a}-0.53\%$
test_sac_speed[False-backward] 11.2962ms 10.7630ms 92.9109 Ops/s 94.3261 Ops/s $\color{#d91a1a}-1.50\%$
test_sac_speed[True-None] 2.8084ms 2.5575ms 391.0107 Ops/s 388.0517 Ops/s $\color{#35bf28}+0.76\%$
test_sac_speed[True-backward] 4.3496ms 4.2632ms 234.5672 Ops/s 232.5425 Ops/s $\color{#35bf28}+0.87\%$
test_sac_speed[reduce-overhead-None] 2.9546ms 2.5828ms 387.1696 Ops/s 387.3195 Ops/s $\color{#d91a1a}-0.04\%$
test_sac_speed[reduce-overhead-backward] 4.3833ms 4.2754ms 233.8965 Ops/s 235.1270 Ops/s $\color{#d91a1a}-0.52\%$
test_redq_speed[False-None] 18.2138ms 13.0548ms 76.6000 Ops/s 75.0685 Ops/s $\color{#35bf28}+2.04\%$
test_redq_speed[False-backward] 30.9069ms 23.7524ms 42.1010 Ops/s 44.4951 Ops/s $\textbf{\color{#d91a1a}-5.38\%}$
test_redq_speed[True-None] 7.5802ms 6.8020ms 147.0154 Ops/s 148.2533 Ops/s $\color{#d91a1a}-0.83\%$
test_redq_speed[True-backward] 14.7858ms 14.3035ms 69.9131 Ops/s 68.9635 Ops/s $\color{#35bf28}+1.38\%$
test_redq_speed[reduce-overhead-None] 8.0571ms 6.8750ms 145.4541 Ops/s 149.4621 Ops/s $\color{#d91a1a}-2.68\%$
test_redq_speed[reduce-overhead-backward] 14.8131ms 14.3977ms 69.4556 Ops/s 68.7856 Ops/s $\color{#35bf28}+0.97\%$
test_redq_deprec_speed[False-None] 15.1380ms 12.9667ms 77.1204 Ops/s 76.7326 Ops/s $\color{#35bf28}+0.51\%$
test_redq_deprec_speed[False-backward] 20.1396ms 19.0029ms 52.6235 Ops/s 52.9267 Ops/s $\color{#d91a1a}-0.57\%$
test_redq_deprec_speed[True-None] 5.8168ms 5.2178ms 191.6505 Ops/s 190.6577 Ops/s $\color{#35bf28}+0.52\%$
test_redq_deprec_speed[True-backward] 10.2924ms 10.0510ms 99.4926 Ops/s 98.5508 Ops/s $\color{#35bf28}+0.96\%$
test_redq_deprec_speed[reduce-overhead-None] 6.4714ms 5.1975ms 192.3991 Ops/s 188.8077 Ops/s $\color{#35bf28}+1.90\%$
test_redq_deprec_speed[reduce-overhead-backward] 10.4503ms 10.0341ms 99.6601 Ops/s 97.3643 Ops/s $\color{#35bf28}+2.36\%$
test_td3_speed[False-None] 8.5342ms 8.0428ms 124.3353 Ops/s 123.3127 Ops/s $\color{#35bf28}+0.83\%$
test_td3_speed[False-backward] 12.4504ms 10.4764ms 95.4526 Ops/s 95.5306 Ops/s $\color{#d91a1a}-0.08\%$
test_td3_speed[True-None] 2.6593ms 2.2933ms 436.0607 Ops/s 437.2310 Ops/s $\color{#d91a1a}-0.27\%$
test_td3_speed[True-backward] 4.0805ms 3.9620ms 252.3965 Ops/s 252.7425 Ops/s $\color{#d91a1a}-0.14\%$
test_td3_speed[reduce-overhead-None] 2.3901ms 2.2650ms 441.5001 Ops/s 439.6735 Ops/s $\color{#35bf28}+0.42\%$
test_td3_speed[reduce-overhead-backward] 3.9817ms 3.9343ms 254.1767 Ops/s 249.7943 Ops/s $\color{#35bf28}+1.75\%$
test_cql_speed[False-None] 38.6340ms 36.6927ms 27.2534 Ops/s 26.8734 Ops/s $\color{#35bf28}+1.41\%$
test_cql_speed[False-backward] 49.4186ms 46.5647ms 21.4755 Ops/s 21.3827 Ops/s $\color{#35bf28}+0.43\%$
test_cql_speed[True-None] 23.9878ms 22.4728ms 44.4983 Ops/s 44.7946 Ops/s $\color{#d91a1a}-0.66\%$
test_cql_speed[True-backward] 30.4834ms 29.0678ms 34.4023 Ops/s 33.2554 Ops/s $\color{#35bf28}+3.45\%$
test_cql_speed[reduce-overhead-None] 23.8518ms 22.3680ms 44.7067 Ops/s 44.8902 Ops/s $\color{#d91a1a}-0.41\%$
test_cql_speed[reduce-overhead-backward] 31.0478ms 29.3294ms 34.0955 Ops/s 33.9215 Ops/s $\color{#35bf28}+0.51\%$
test_a2c_speed[False-None] 8.6992ms 7.2070ms 138.7545 Ops/s 138.7350 Ops/s $\color{#35bf28}+0.01\%$
test_a2c_speed[False-backward] 15.2013ms 14.4857ms 69.0338 Ops/s 69.6258 Ops/s $\color{#d91a1a}-0.85\%$
test_a2c_speed[True-None] 5.5250ms 4.6558ms 214.7860 Ops/s 212.1512 Ops/s $\color{#35bf28}+1.24\%$
test_a2c_speed[True-backward] 11.5073ms 11.0795ms 90.2570 Ops/s 89.9367 Ops/s $\color{#35bf28}+0.36\%$
test_a2c_speed[reduce-overhead-None] 5.2296ms 4.6459ms 215.2420 Ops/s 214.4517 Ops/s $\color{#35bf28}+0.37\%$
test_a2c_speed[reduce-overhead-backward] 11.7039ms 11.0987ms 90.1006 Ops/s 89.1317 Ops/s $\color{#35bf28}+1.09\%$
test_ppo_speed[False-None] 8.2007ms 7.4618ms 134.0167 Ops/s 129.3428 Ops/s $\color{#35bf28}+3.61\%$
test_ppo_speed[False-backward] 16.3562ms 14.7057ms 68.0008 Ops/s 68.1100 Ops/s $\color{#d91a1a}-0.16\%$
test_ppo_speed[True-None] 5.9615ms 5.0427ms 198.3059 Ops/s 197.4417 Ops/s $\color{#35bf28}+0.44\%$
test_ppo_speed[True-backward] 12.0918ms 10.9714ms 91.1459 Ops/s 90.8013 Ops/s $\color{#35bf28}+0.38\%$
test_ppo_speed[reduce-overhead-None] 5.6890ms 5.0151ms 199.3961 Ops/s 197.9292 Ops/s $\color{#35bf28}+0.74\%$
test_ppo_speed[reduce-overhead-backward] 11.7284ms 10.9799ms 91.0758 Ops/s 90.7414 Ops/s $\color{#35bf28}+0.37\%$
test_reinforce_speed[False-None] 7.7593ms 6.5376ms 152.9605 Ops/s 152.8652 Ops/s $\color{#35bf28}+0.06\%$
test_reinforce_speed[False-backward] 13.7384ms 10.2661ms 97.4081 Ops/s 101.9673 Ops/s $\color{#d91a1a}-4.47\%$
test_reinforce_speed[True-None] 4.7429ms 4.0432ms 247.3262 Ops/s 247.1602 Ops/s $\color{#35bf28}+0.07\%$
test_reinforce_speed[True-backward] 10.4313ms 9.9674ms 100.3268 Ops/s 100.7774 Ops/s $\color{#d91a1a}-0.45\%$
test_reinforce_speed[reduce-overhead-None] 4.7050ms 4.0525ms 246.7611 Ops/s 244.7275 Ops/s $\color{#35bf28}+0.83\%$
test_reinforce_speed[reduce-overhead-backward] 10.4095ms 9.9933ms 100.0672 Ops/s 91.7373 Ops/s $\textbf{\color{#35bf28}+9.08\%}$
test_iql_speed[False-None] 33.1305ms 32.3606ms 30.9018 Ops/s 30.0077 Ops/s $\color{#35bf28}+2.98\%$
test_iql_speed[False-backward] 47.2542ms 45.5798ms 21.9396 Ops/s 21.1763 Ops/s $\color{#35bf28}+3.60\%$
test_iql_speed[True-None] 16.9929ms 15.8546ms 63.0731 Ops/s 62.2874 Ops/s $\color{#35bf28}+1.26\%$
test_iql_speed[True-backward] 27.4699ms 26.7744ms 37.3491 Ops/s 34.8635 Ops/s $\textbf{\color{#35bf28}+7.13\%}$
test_iql_speed[reduce-overhead-None] 17.0869ms 15.7279ms 63.5812 Ops/s 62.3140 Ops/s $\color{#35bf28}+2.03\%$
test_iql_speed[reduce-overhead-backward] 27.5168ms 26.8002ms 37.3131 Ops/s 36.7836 Ops/s $\color{#35bf28}+1.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3019ms 4.7684ms 209.7135 Ops/s 212.7121 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7409ms 0.5127ms 1.9504 KOps/s 1.9512 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7737ms 0.4877ms 2.0506 KOps/s 2.0310 KOps/s $\color{#35bf28}+0.97\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1582ms 4.4934ms 222.5484 Ops/s 221.9393 Ops/s $\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5253ms 0.5009ms 1.9963 KOps/s 1.9795 KOps/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7238ms 0.4837ms 2.0674 KOps/s 2.0926 KOps/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.1802ms 1.6524ms 605.1806 Ops/s 604.9194 Ops/s $\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0759ms 1.5685ms 637.5571 Ops/s 639.1389 Ops/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.2814ms 4.6023ms 217.2827 Ops/s 213.9716 Ops/s $\color{#35bf28}+1.55\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7739ms 0.6490ms 1.5408 KOps/s 1.5552 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8620ms 0.6253ms 1.5993 KOps/s 1.6088 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.2354ms 4.4668ms 223.8743 Ops/s 218.5744 Ops/s $\color{#35bf28}+2.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8928ms 0.5104ms 1.9593 KOps/s 1.9426 KOps/s $\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8262ms 0.4996ms 2.0015 KOps/s 2.0190 KOps/s $\color{#d91a1a}-0.87\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.7657ms 4.4112ms 226.6952 Ops/s 221.6259 Ops/s $\color{#35bf28}+2.29\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9219ms 0.5041ms 1.9836 KOps/s 2.0000 KOps/s $\color{#d91a1a}-0.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7198ms 0.4802ms 2.0823 KOps/s 2.0838 KOps/s $\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.3028ms 4.5749ms 218.5818 Ops/s 216.1934 Ops/s $\color{#35bf28}+1.10\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0905ms 0.6516ms 1.5346 KOps/s 1.5436 KOps/s $\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9123ms 0.6230ms 1.6052 KOps/s 1.6094 KOps/s $\color{#d91a1a}-0.27\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.8631s 21.2888ms 46.9730 Ops/s 239.7104 Ops/s $\textbf{\color{#d91a1a}-80.40\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.1989ms 2.3454ms 426.3707 Ops/s 460.0480 Ops/s $\textbf{\color{#d91a1a}-7.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.9221ms 1.3956ms 716.5506 Ops/s 770.8333 Ops/s $\textbf{\color{#d91a1a}-7.04\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 5.6291ms 4.2537ms 235.0869 Ops/s 24.6221 Ops/s $\textbf{\color{#35bf28}+854.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.8995ms 2.3864ms 419.0466 Ops/s 437.5894 Ops/s $\color{#d91a1a}-4.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.7480ms 1.3650ms 732.6211 Ops/s 767.6951 Ops/s $\color{#d91a1a}-4.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 5.9456ms 4.3643ms 229.1319 Ops/s 226.6757 Ops/s $\color{#35bf28}+1.08\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 5.9920ms 2.4083ms 415.2379 Ops/s 408.7209 Ops/s $\color{#35bf28}+1.59\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.5307ms 1.6141ms 619.5507 Ops/s 645.1721 Ops/s $\color{#d91a1a}-3.97\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 12.2349ms 11.7745ms 84.9290 Ops/s 81.6651 Ops/s $\color{#35bf28}+4.00\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.4416ms 14.3684ms 69.5970 Ops/s 70.3694 Ops/s $\color{#d91a1a}-1.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 21.7775ms 20.6006ms 48.5422 Ops/s 47.8666 Ops/s $\color{#35bf28}+1.41\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.4912ms 14.3678ms 69.5999 Ops/s 69.7351 Ops/s $\color{#d91a1a}-0.19\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 21.9413ms 20.6947ms 48.3215 Ops/s 48.2439 Ops/s $\color{#35bf28}+0.16\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 18.6359ms 15.7223ms 63.6038 Ops/s 64.2558 Ops/s $\color{#d91a1a}-1.01\%$

Update
7370f08 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 6, 2025
[Refactor] LLM data structures
c7b4477 
ghstack-source-id: 32f6796
Pull Request resolved: #2834
Update
cb3f44a 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 6, 2025
[Refactor] LLM data structures
69e2254 
ghstack-source-id: 3f928dd
Pull Request resolved: #2834
Update
d154b5e 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Mar 6, 2025
[Refactor] LLM data structures
44e21bc 
ghstack-source-id: 8483fe0
Pull Request resolved: #2834
@vmoens vmoens merged commit d154b5e into gh/vmoens/107/base Mar 6, 2025
20 of 31 checks passed
vmoens pushed a commit that referenced this pull request Mar 6, 2025
[Refactor] LLM data structures
73c7b0a 
ghstack-source-id: 8483fe0
Pull Request resolved: #2834
@vmoens vmoens deleted the gh/vmoens/107/head branch March 6, 2025 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens @facebook-github-bot