Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Feature] PPO Trainer Updates #3190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 29 commits into
base: gh/vmoens/153/base
Choose a base branch
from
Open

[Feature] PPO Trainer Updates #3190

wants to merge 29 commits into from

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
5d33073 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3190

Note: Links to docs will display an error until the docs builds have been completed.

❌ 12 New Failures, 1 Unrelated Failure

As of commit 3cf8395 with merge base 963fdd4 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
8557795 
[ghstack-poisoned]
@vmoens
Update
8e5cce3 
[ghstack-poisoned]
@vmoens
Update
3c22f43 
[ghstack-poisoned]
@vmoens
Update
308cd5f 
[ghstack-poisoned]
@vmoens
Update
7db41a4 
[ghstack-poisoned]
@vmoens
Update
dc7193a 
[ghstack-poisoned]
@vmoens
Update
717bc3f 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
6072106 
[ghstack-poisoned]
@vmoens
Update
7356e29 
[ghstack-poisoned]
@vmoens
Update
7f4c006 
[ghstack-poisoned]
@vmoens
Update
ddcc7df 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
8ef7b80 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
5a672ad 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
91e6c02 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}14$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 85.3353μs 81.9813μs 12.1979 KOps/s 12.6877 KOps/s $\color{#d91a1a}-3.86\%$
test_tensor_to_bytestream_speed[torch.save] 0.1425ms 0.1419ms 7.0478 KOps/s 7.3381 KOps/s $\color{#d91a1a}-3.96\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1276s 0.1264s 7.9097 Ops/s 8.7748 Ops/s $\textbf{\color{#d91a1a}-9.86\%}$
test_tensor_to_bytestream_speed[numpy] 2.8740μs 2.8694μs 348.5028 KOps/s 352.6711 KOps/s $\color{#d91a1a}-1.18\%$
test_tensor_to_bytestream_speed[safetensors] 41.4357μs 41.1557μs 24.2980 KOps/s 24.0728 KOps/s $\color{#35bf28}+0.94\%$
test_simple 0.5401s 0.5394s 1.8541 Ops/s 1.7916 Ops/s $\color{#35bf28}+3.49\%$
test_transformed 1.2104s 1.1081s 0.9025 Ops/s 0.9052 Ops/s $\color{#d91a1a}-0.30\%$
test_serial 1.6269s 1.6191s 0.6176 Ops/s 0.6093 Ops/s $\color{#35bf28}+1.36\%$
test_parallel 1.1101s 1.0675s 0.9368 Ops/s 0.9079 Ops/s $\color{#35bf28}+3.18\%$
test_step_mdp_speed[True-True-True-True-True] 0.2159ms 43.4325μs 23.0242 KOps/s 22.7188 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-True-False] 49.8910μs 24.4600μs 40.8831 KOps/s 41.9640 KOps/s $\color{#d91a1a}-2.58\%$
test_step_mdp_speed[True-True-True-False-True] 56.8800μs 24.8941μs 40.1702 KOps/s 40.6125 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-True-False-False] 42.5910μs 13.5725μs 73.6783 KOps/s 75.9873 KOps/s $\color{#d91a1a}-3.04\%$
test_step_mdp_speed[True-True-False-True-True] 75.9810μs 46.5870μs 21.4652 KOps/s 21.6450 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-False-True-False] 65.8010μs 26.9487μs 37.1075 KOps/s 37.6395 KOps/s $\color{#d91a1a}-1.41\%$
test_step_mdp_speed[True-True-False-False-True] 99.2520μs 27.1829μs 36.7878 KOps/s 36.8262 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-True-False-False-False] 45.4810μs 16.5703μs 60.3489 KOps/s 61.9783 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[True-False-True-True-True] 92.5910μs 49.3802μs 20.2511 KOps/s 20.3845 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-True-True-False] 66.6010μs 29.7993μs 33.5579 KOps/s 33.9913 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-True-False-True] 59.0610μs 27.4229μs 36.4658 KOps/s 35.8049 KOps/s $\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-False-True-False-False] 75.6210μs 16.2415μs 61.5708 KOps/s 61.3704 KOps/s $\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-False-False-True-True] 76.6420μs 51.3143μs 19.4877 KOps/s 19.3613 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-False-False-True-False] 67.9110μs 32.4461μs 30.8203 KOps/s 30.6368 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-False-False-True] 64.7210μs 29.6879μs 33.6838 KOps/s 33.6253 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-False-False-False-False] 49.2810μs 18.7994μs 53.1932 KOps/s 53.6762 KOps/s $\color{#d91a1a}-0.90\%$
test_step_mdp_speed[False-True-True-True-True] 77.4320μs 48.7509μs 20.5124 KOps/s 20.3781 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-True-True-True-False] 60.7510μs 29.5246μs 33.8701 KOps/s 34.0029 KOps/s $\color{#d91a1a}-0.39\%$
test_step_mdp_speed[False-True-True-False-True] 2.5542ms 31.4389μs 31.8077 KOps/s 31.7275 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-True-True-False-False] 44.9910μs 18.0858μs 55.2920 KOps/s 55.4723 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-False-True-True] 84.2010μs 51.4551μs 19.4344 KOps/s 19.6596 KOps/s $\color{#d91a1a}-1.15\%$
test_step_mdp_speed[False-True-False-True-False] 71.4010μs 32.4606μs 30.8066 KOps/s 31.1991 KOps/s $\color{#d91a1a}-1.26\%$
test_step_mdp_speed[False-True-False-False-True] 65.8410μs 33.3270μs 30.0057 KOps/s 29.6876 KOps/s $\color{#35bf28}+1.07\%$
test_step_mdp_speed[False-True-False-False-False] 52.2100μs 20.7139μs 48.2767 KOps/s 48.6630 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-False-True-True-True] 90.4110μs 55.1915μs 18.1187 KOps/s 18.6290 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-False-True-True-False] 62.1410μs 35.2521μs 28.3671 KOps/s 28.8650 KOps/s $\color{#d91a1a}-1.72\%$
test_step_mdp_speed[False-False-True-False-True] 65.8700μs 33.7768μs 29.6061 KOps/s 30.1717 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-False-True-False-False] 47.6110μs 20.7092μs 48.2877 KOps/s 49.1890 KOps/s $\color{#d91a1a}-1.83\%$
test_step_mdp_speed[False-False-False-True-True] 87.4410μs 55.8454μs 17.9066 KOps/s 17.6476 KOps/s $\color{#35bf28}+1.47\%$
test_step_mdp_speed[False-False-False-True-False] 68.2210μs 36.9983μs 27.0283 KOps/s 26.6874 KOps/s $\color{#35bf28}+1.28\%$
test_step_mdp_speed[False-False-False-False-True] 68.4400μs 35.2807μs 28.3441 KOps/s 27.5530 KOps/s $\color{#35bf28}+2.87\%$
test_step_mdp_speed[False-False-False-False-False] 51.1300μs 22.8199μs 43.8215 KOps/s 43.3706 KOps/s $\color{#35bf28}+1.04\%$
test_values[generalized_advantage_estimate-True-True] 10.0519ms 9.5504ms 104.7080 Ops/s 104.2537 Ops/s $\color{#35bf28}+0.44\%$
test_values[vec_generalized_advantage_estimate-True-True] 20.7077ms 17.8262ms 56.0971 Ops/s 57.0288 Ops/s $\color{#d91a1a}-1.63\%$
test_values[td0_return_estimate-False-False] 3.2679ms 0.1903ms 5.2553 KOps/s 7.5800 KOps/s $\textbf{\color{#d91a1a}-30.67\%}$
test_values[td1_return_estimate-False-False] 26.9421ms 26.2844ms 38.0454 Ops/s 38.7191 Ops/s $\color{#d91a1a}-1.74\%$
test_values[vec_td1_return_estimate-False-False] 18.2143ms 17.6560ms 56.6381 Ops/s 56.6055 Ops/s $\color{#35bf28}+0.06\%$
test_values[td_lambda_return_estimate-True-False] 40.2932ms 39.2490ms 25.4783 Ops/s 24.0089 Ops/s $\textbf{\color{#35bf28}+6.12\%}$
test_values[vec_td_lambda_return_estimate-True-False] 20.8137ms 17.6947ms 56.5140 Ops/s 56.6517 Ops/s $\color{#d91a1a}-0.24\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.7653ms 8.5855ms 116.4756 Ops/s 114.8869 Ops/s $\color{#35bf28}+1.38\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 5.0769ms 1.5233ms 656.4910 Ops/s 647.1700 Ops/s $\color{#35bf28}+1.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5245ms 0.4046ms 2.4715 KOps/s 2.5588 KOps/s $\color{#d91a1a}-3.41\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.7459ms 34.1630ms 29.2715 Ops/s 28.6859 Ops/s $\color{#35bf28}+2.04\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1340ms 1.7143ms 583.3221 Ops/s 590.8451 Ops/s $\color{#d91a1a}-1.27\%$
test_dqn_speed[False-None] 6.6259ms 1.3921ms 718.3375 Ops/s 732.1942 Ops/s $\color{#d91a1a}-1.89\%$
test_dqn_speed[False-backward] 1.9823ms 1.9133ms 522.6570 Ops/s 537.0322 Ops/s $\color{#d91a1a}-2.68\%$
test_dqn_speed[True-None] 0.9109ms 0.5041ms 1.9836 KOps/s 1.9436 KOps/s $\color{#35bf28}+2.06\%$
test_dqn_speed[True-backward] 1.0302ms 0.9474ms 1.0555 KOps/s 934.9680 Ops/s $\textbf{\color{#35bf28}+12.89\%}$
test_dqn_speed[reduce-overhead-None] 0.6126ms 0.4977ms 2.0092 KOps/s 1.9088 KOps/s $\textbf{\color{#35bf28}+5.26\%}$
test_dqn_speed[reduce-overhead-backward] 1.0188ms 0.9431ms 1.0603 KOps/s 952.7928 Ops/s $\textbf{\color{#35bf28}+11.28\%}$
test_ddpg_speed[False-None] 3.2110ms 2.8293ms 353.4476 Ops/s 350.1290 Ops/s $\color{#35bf28}+0.95\%$
test_ddpg_speed[False-backward] 4.2014ms 4.0344ms 247.8710 Ops/s 251.2598 Ops/s $\color{#d91a1a}-1.35\%$
test_ddpg_speed[True-None] 1.5324ms 1.3556ms 737.7058 Ops/s 728.6050 Ops/s $\color{#35bf28}+1.25\%$
test_ddpg_speed[True-backward] 2.3798ms 2.3314ms 428.9226 Ops/s 409.1529 Ops/s $\color{#35bf28}+4.83\%$
test_ddpg_speed[reduce-overhead-None] 1.4716ms 1.3648ms 732.6846 Ops/s 716.2347 Ops/s $\color{#35bf28}+2.30\%$
test_ddpg_speed[reduce-overhead-backward] 2.3863ms 2.3351ms 428.2520 Ops/s 350.9816 Ops/s $\textbf{\color{#35bf28}+22.02\%}$
test_sac_speed[False-None] 8.3103ms 7.7419ms 129.1665 Ops/s 109.2147 Ops/s $\textbf{\color{#35bf28}+18.27\%}$
test_sac_speed[False-backward] 11.2744ms 10.9038ms 91.7114 Ops/s 91.3598 Ops/s $\color{#35bf28}+0.38\%$
test_sac_speed[True-None] 2.3820ms 2.1123ms 473.4225 Ops/s 468.9348 Ops/s $\color{#35bf28}+0.96\%$
test_sac_speed[True-backward] 4.2583ms 4.0220ms 248.6319 Ops/s 247.9544 Ops/s $\color{#35bf28}+0.27\%$
test_sac_speed[reduce-overhead-None] 2.4907ms 2.1059ms 474.8635 Ops/s 463.5216 Ops/s $\color{#35bf28}+2.45\%$
test_sac_speed[reduce-overhead-backward] 4.1809ms 4.0317ms 248.0367 Ops/s 240.4347 Ops/s $\color{#35bf28}+3.16\%$
test_redq_speed[False-None] 13.9551ms 10.2578ms 97.4867 Ops/s 96.9618 Ops/s $\color{#35bf28}+0.54\%$
test_redq_speed[False-backward] 18.8782ms 17.6715ms 56.5884 Ops/s 53.4295 Ops/s $\textbf{\color{#35bf28}+5.91\%}$
test_redq_speed[True-None] 5.0353ms 4.4702ms 223.7055 Ops/s 222.8746 Ops/s $\color{#35bf28}+0.37\%$
test_redq_speed[True-backward] 10.2623ms 10.0383ms 99.6182 Ops/s 98.5057 Ops/s $\color{#35bf28}+1.13\%$
test_redq_speed[reduce-overhead-None] 4.6463ms 4.4447ms 224.9874 Ops/s 231.9082 Ops/s $\color{#d91a1a}-2.98\%$
test_redq_speed[reduce-overhead-backward] 10.4032ms 10.1349ms 98.6688 Ops/s 100.6169 Ops/s $\color{#d91a1a}-1.94\%$
test_redq_deprec_speed[False-None] 11.4987ms 10.8660ms 92.0306 Ops/s 90.4184 Ops/s $\color{#35bf28}+1.78\%$
test_redq_deprec_speed[False-backward] 16.7043ms 15.8650ms 63.0317 Ops/s 63.0921 Ops/s $\color{#d91a1a}-0.10\%$
test_redq_deprec_speed[True-None] 4.0496ms 3.6405ms 274.6886 Ops/s 269.2272 Ops/s $\color{#35bf28}+2.03\%$
test_redq_deprec_speed[True-backward] 8.1935ms 7.7773ms 128.5786 Ops/s 120.1821 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_redq_deprec_speed[reduce-overhead-None] 4.4152ms 3.5801ms 279.3237 Ops/s 272.1900 Ops/s $\color{#35bf28}+2.62\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.1201ms 7.6990ms 129.8867 Ops/s 127.1463 Ops/s $\color{#35bf28}+2.16\%$
test_td3_speed[False-None] 7.8852ms 7.7329ms 129.3177 Ops/s 129.9919 Ops/s $\color{#d91a1a}-0.52\%$
test_td3_speed[False-backward] 11.0653ms 10.6553ms 93.8502 Ops/s 94.9530 Ops/s $\color{#d91a1a}-1.16\%$
test_td3_speed[True-None] 1.8770ms 1.8226ms 548.6564 Ops/s 553.9987 Ops/s $\color{#d91a1a}-0.96\%$
test_td3_speed[True-backward] 3.7319ms 3.5863ms 278.8421 Ops/s 262.6978 Ops/s $\textbf{\color{#35bf28}+6.15\%}$
test_td3_speed[reduce-overhead-None] 1.8571ms 1.7804ms 561.6807 Ops/s 556.9314 Ops/s $\color{#35bf28}+0.85\%$
test_td3_speed[reduce-overhead-backward] 4.0553ms 3.6301ms 275.4754 Ops/s 240.6399 Ops/s $\textbf{\color{#35bf28}+14.48\%}$
test_cql_speed[False-None] 29.1350ms 26.0219ms 38.4292 Ops/s 39.0131 Ops/s $\color{#d91a1a}-1.50\%$
test_cql_speed[False-backward] 39.3545ms 35.6469ms 28.0529 Ops/s 28.5132 Ops/s $\color{#d91a1a}-1.61\%$
test_cql_speed[True-None] 12.8565ms 12.3874ms 80.7274 Ops/s 79.0530 Ops/s $\color{#35bf28}+2.12\%$
test_cql_speed[True-backward] 18.7825ms 18.4190ms 54.2917 Ops/s 55.2273 Ops/s $\color{#d91a1a}-1.69\%$
test_cql_speed[reduce-overhead-None] 13.0129ms 12.5516ms 79.6709 Ops/s 79.9566 Ops/s $\color{#d91a1a}-0.36\%$
test_cql_speed[reduce-overhead-backward] 19.6992ms 18.7298ms 53.3909 Ops/s 55.0476 Ops/s $\color{#d91a1a}-3.01\%$
test_a2c_speed[False-None] 5.8351ms 5.4155ms 184.6553 Ops/s 181.8903 Ops/s $\color{#35bf28}+1.52\%$
test_a2c_speed[False-backward] 12.5496ms 11.9947ms 83.3704 Ops/s 83.8044 Ops/s $\color{#d91a1a}-0.52\%$
test_a2c_speed[True-None] 4.1243ms 3.7343ms 267.7855 Ops/s 261.2807 Ops/s $\color{#35bf28}+2.49\%$
test_a2c_speed[True-backward] 8.9234ms 8.5862ms 116.4663 Ops/s 113.9816 Ops/s $\color{#35bf28}+2.18\%$
test_a2c_speed[reduce-overhead-None] 4.2004ms 3.7326ms 267.9118 Ops/s 267.8361 Ops/s $\color{#35bf28}+0.03\%$
test_a2c_speed[reduce-overhead-backward] 9.4686ms 8.9796ms 111.3636 Ops/s 112.3482 Ops/s $\color{#d91a1a}-0.88\%$
test_ppo_speed[False-None] 6.1894ms 5.7909ms 172.6856 Ops/s 170.2289 Ops/s $\color{#35bf28}+1.44\%$
test_ppo_speed[False-backward] 12.8092ms 12.5719ms 79.5424 Ops/s 80.4711 Ops/s $\color{#d91a1a}-1.15\%$
test_ppo_speed[True-None] 4.1026ms 3.6573ms 273.4288 Ops/s 267.7317 Ops/s $\color{#35bf28}+2.13\%$
test_ppo_speed[True-backward] 9.0072ms 8.5874ms 116.4492 Ops/s 112.1554 Ops/s $\color{#35bf28}+3.83\%$
test_ppo_speed[reduce-overhead-None] 4.1107ms 3.6559ms 273.5272 Ops/s 272.0950 Ops/s $\color{#35bf28}+0.53\%$
test_ppo_speed[reduce-overhead-backward] 9.0730ms 8.8418ms 113.0998 Ops/s 112.3291 Ops/s $\color{#35bf28}+0.69\%$
test_reinforce_speed[False-None] 5.0004ms 4.5871ms 218.0029 Ops/s 218.5874 Ops/s $\color{#d91a1a}-0.27\%$
test_reinforce_speed[False-backward] 7.6919ms 7.4664ms 133.9330 Ops/s 135.2093 Ops/s $\color{#d91a1a}-0.94\%$
test_reinforce_speed[True-None] 3.2648ms 2.8873ms 346.3452 Ops/s 343.7893 Ops/s $\color{#35bf28}+0.74\%$
test_reinforce_speed[True-backward] 8.2444ms 7.7477ms 129.0708 Ops/s 129.1448 Ops/s $\color{#d91a1a}-0.06\%$
test_reinforce_speed[reduce-overhead-None] 3.3378ms 2.8696ms 348.4836 Ops/s 334.6853 Ops/s $\color{#35bf28}+4.12\%$
test_reinforce_speed[reduce-overhead-backward] 8.3067ms 7.9820ms 125.2813 Ops/s 122.5164 Ops/s $\color{#35bf28}+2.26\%$
test_iql_speed[False-None] 24.9790ms 19.9460ms 50.1354 Ops/s 47.8746 Ops/s $\color{#35bf28}+4.72\%$
test_iql_speed[False-backward] 34.7614ms 30.5333ms 32.7511 Ops/s 32.0489 Ops/s $\color{#35bf28}+2.19\%$
test_iql_speed[True-None] 9.1204ms 8.5939ms 116.3618 Ops/s 115.4821 Ops/s $\color{#35bf28}+0.76\%$
test_iql_speed[True-backward] 17.5332ms 17.0436ms 58.6730 Ops/s 55.7303 Ops/s $\textbf{\color{#35bf28}+5.28\%}$
test_iql_speed[reduce-overhead-None] 9.0988ms 8.6178ms 116.0387 Ops/s 115.6490 Ops/s $\color{#35bf28}+0.34\%$
test_iql_speed[reduce-overhead-backward] 18.0133ms 17.3776ms 57.5454 Ops/s 58.0658 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.1456ms 5.7757ms 173.1405 Ops/s 172.4039 Ops/s $\color{#35bf28}+0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6366ms 0.3126ms 3.1992 KOps/s 3.1598 KOps/s $\color{#35bf28}+1.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6830ms 0.2567ms 3.8952 KOps/s 3.3261 KOps/s $\textbf{\color{#35bf28}+17.11\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8676ms 5.5669ms 179.6332 Ops/s 181.8161 Ops/s $\color{#d91a1a}-1.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9647ms 0.3182ms 3.1432 KOps/s 3.2178 KOps/s $\color{#d91a1a}-2.32\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6207ms 0.3142ms 3.1823 KOps/s 3.4038 KOps/s $\textbf{\color{#d91a1a}-6.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6189ms 1.3966ms 716.0401 Ops/s 804.7433 Ops/s $\textbf{\color{#d91a1a}-11.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5888ms 1.3234ms 755.6043 Ops/s 826.6962 Ops/s $\textbf{\color{#d91a1a}-8.60\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 12.5529ms 5.8850ms 169.9233 Ops/s 176.9096 Ops/s $\color{#d91a1a}-3.95\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1590ms 0.5119ms 1.9536 KOps/s 2.1690 KOps/s $\textbf{\color{#d91a1a}-9.93\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7859ms 0.4936ms 2.0260 KOps/s 2.2314 KOps/s $\textbf{\color{#d91a1a}-9.20\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7052ms 5.5287ms 180.8737 Ops/s 181.5456 Ops/s $\color{#d91a1a}-0.37\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7600ms 0.3378ms 2.9606 KOps/s 836.9176 Ops/s $\textbf{\color{#35bf28}+253.75\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6551ms 0.3311ms 3.0200 KOps/s 3.3752 KOps/s $\textbf{\color{#d91a1a}-10.53\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.6953ms 5.4892ms 182.1765 Ops/s 178.5458 Ops/s $\color{#35bf28}+2.03\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.7763ms 0.3655ms 2.7356 KOps/s 3.1258 KOps/s $\textbf{\color{#d91a1a}-12.48\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6028ms 0.3484ms 2.8700 KOps/s 3.7863 KOps/s $\textbf{\color{#d91a1a}-24.20\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.8954ms 5.6512ms 176.9547 Ops/s 173.4363 Ops/s $\color{#35bf28}+2.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0146ms 0.5134ms 1.9477 KOps/s 2.1899 KOps/s $\textbf{\color{#d91a1a}-11.06\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6991ms 0.4971ms 2.0118 KOps/s 2.3116 KOps/s $\textbf{\color{#d91a1a}-12.97\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.6176ms 4.9672ms 201.3217 Ops/s 196.7566 Ops/s $\color{#35bf28}+2.32\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.6939ms 2.2971ms 435.3345 Ops/s 432.1575 Ops/s $\color{#35bf28}+0.74\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.8349ms 1.2567ms 795.7244 Ops/s 865.5440 Ops/s $\textbf{\color{#d91a1a}-8.07\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5098s 15.0866ms 66.2841 Ops/s 54.9213 Ops/s $\textbf{\color{#35bf28}+20.69\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.1222ms 1.9406ms 515.3148 Ops/s 590.2844 Ops/s $\textbf{\color{#d91a1a}-12.70\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.6256ms 1.2040ms 830.5682 Ops/s 805.4198 Ops/s $\color{#35bf28}+3.12\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.9366ms 5.1662ms 193.5653 Ops/s 189.6489 Ops/s $\color{#35bf28}+2.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 9.2710ms 2.1909ms 456.4312 Ops/s 460.7772 Ops/s $\color{#d91a1a}-0.94\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.1570ms 1.3489ms 741.3263 Ops/s 730.5436 Ops/s $\color{#35bf28}+1.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 34.5497ms 32.3972ms 30.8669 Ops/s 30.4164 Ops/s $\color{#35bf28}+1.48\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.8008ms 17.1077ms 58.4531 Ops/s 59.0236 Ops/s $\color{#d91a1a}-0.97\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 35.3120ms 33.3742ms 29.9632 Ops/s 29.6597 Ops/s $\color{#35bf28}+1.02\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.5804ms 17.7688ms 56.2783 Ops/s 57.4012 Ops/s $\color{#d91a1a}-1.96\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 36.5468ms 34.9356ms 28.6241 Ops/s 28.3322 Ops/s $\color{#35bf28}+1.03\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.2407ms 19.0597ms 52.4667 Ops/s 54.0710 Ops/s $\color{#d91a1a}-2.97\%$

vmoens added 7 commits October 19, 2025 20:56
@vmoens
Update
09a93f3 
[ghstack-poisoned]
@vmoens
Update
f523683 
[ghstack-poisoned]
@vmoens
Update
c3b011f 
[ghstack-poisoned]
@vmoens
Update
647fba0 
[ghstack-poisoned]
@vmoens
Update
0978ca8 
[ghstack-poisoned]
@vmoens
Update
7759c3d 
[ghstack-poisoned]
@vmoens
Update
73bc5de 
[ghstack-poisoned]
This was referenced Oct 22, 2025
Open
Open
vmoens added 7 commits October 22, 2025 12:31
@vmoens
Update
2915075 
[ghstack-poisoned]
@vmoens
Update
a106e2f 
[ghstack-poisoned]
@vmoens
Update
2f51940 
[ghstack-poisoned]
@vmoens
Update
b9bb0a4 
[ghstack-poisoned]
@vmoens
Update
c71aaad 
[ghstack-poisoned]
@vmoens
Update
4438cdd 
[ghstack-poisoned]
@vmoens
Update
3cf8395 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens