Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Feature] SAC Trainer #3191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 33 commits into
base: gh/vmoens/154/base
Choose a base branch
from
Open

[Feature] SAC Trainer #3191

wants to merge 33 commits into from
+4 −0

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
4d6443e 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3191

Note: Links to docs will display an error until the docs builds have been completed.

❌ 10 New Failures, 1 Unrelated Failure

As of commit 6f4fc6d with merge base 01d2801 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
ec4e903 
[ghstack-poisoned]
@vmoens
Update
58b15b5 
[ghstack-poisoned]
@vmoens
Update
3e42d1f 
[ghstack-poisoned]
@vmoens
Update
3ffb582 
[ghstack-poisoned]
@vmoens
Update
b0b8b39 
[ghstack-poisoned]
@vmoens
Update
3b61b27 
[ghstack-poisoned]
@vmoens
Update
fd9dec0 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Merged
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
26026d4 
[ghstack-poisoned]
@vmoens
Update
a411944 
[ghstack-poisoned]
@vmoens
Update
7739792 
[ghstack-poisoned]
@vmoens
Update
f9a29c8 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
579b61b 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
a3a0abc 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
072605a 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}7$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 83.1333μs 82.0127μs 12.1932 KOps/s 11.9280 KOps/s $\color{#35bf28}+2.22\%$
test_tensor_to_bytestream_speed[torch.save] 0.1429ms 0.1411ms 7.0847 KOps/s 6.9510 KOps/s $\color{#35bf28}+1.92\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1261s 0.1258s 7.9507 Ops/s 8.2599 Ops/s $\color{#d91a1a}-3.74\%$
test_tensor_to_bytestream_speed[numpy] 2.7931μs 2.7906μs 358.3479 KOps/s 357.0929 KOps/s $\color{#35bf28}+0.35\%$
test_tensor_to_bytestream_speed[safetensors] 44.1955μs 43.2471μs 23.1230 KOps/s 24.1846 KOps/s $\color{#d91a1a}-4.39\%$
test_simple 0.6616s 0.5725s 1.7466 Ops/s 1.7445 Ops/s $\color{#35bf28}+0.12\%$
test_transformed 1.2178s 1.1261s 0.8880 Ops/s 0.8857 Ops/s $\color{#35bf28}+0.26\%$
test_serial 1.6643s 1.6620s 0.6017 Ops/s 0.5919 Ops/s $\color{#35bf28}+1.66\%$
test_parallel 1.1662s 1.0905s 0.9170 Ops/s 0.9130 Ops/s $\color{#35bf28}+0.44\%$
test_step_mdp_speed[True-True-True-True-True] 0.1777ms 44.6680μs 22.3874 KOps/s 22.6641 KOps/s $\color{#d91a1a}-1.22\%$
test_step_mdp_speed[True-True-True-True-False] 58.7610μs 25.7998μs 38.7600 KOps/s 38.2544 KOps/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-False-True] 53.5810μs 25.7958μs 38.7661 KOps/s 38.8901 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-True-False-False] 42.5510μs 13.8675μs 72.1112 KOps/s 66.8002 KOps/s $\textbf{\color{#35bf28}+7.95\%}$
test_step_mdp_speed[True-True-False-True-True] 84.7110μs 48.9425μs 20.4321 KOps/s 20.4108 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-True-False-True-False] 56.7510μs 28.6338μs 34.9238 KOps/s 34.7073 KOps/s $\color{#35bf28}+0.62\%$
test_step_mdp_speed[True-True-False-False-True] 65.4010μs 28.4953μs 35.0935 KOps/s 33.4329 KOps/s $\color{#35bf28}+4.97\%$
test_step_mdp_speed[True-True-False-False-False] 60.8610μs 16.8027μs 59.5143 KOps/s 58.9918 KOps/s $\color{#35bf28}+0.89\%$
test_step_mdp_speed[True-False-True-True-True] 90.2820μs 51.7861μs 19.3102 KOps/s 19.4040 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-True-False] 0.1409ms 31.1429μs 32.1100 KOps/s 32.0303 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-False-True-False-True] 56.3800μs 28.5969μs 34.9688 KOps/s 33.5054 KOps/s $\color{#35bf28}+4.37\%$
test_step_mdp_speed[True-False-True-False-False] 44.8510μs 16.8246μs 59.4368 KOps/s 57.0987 KOps/s $\color{#35bf28}+4.09\%$
test_step_mdp_speed[True-False-False-True-True] 94.5820μs 54.1742μs 18.4590 KOps/s 17.6875 KOps/s $\color{#35bf28}+4.36\%$
test_step_mdp_speed[True-False-False-True-False] 67.2210μs 33.8022μs 29.5839 KOps/s 29.0787 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[True-False-False-False-True] 60.0710μs 31.0671μs 32.1884 KOps/s 32.4890 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-False-False-False-False] 55.0910μs 19.5831μs 51.0643 KOps/s 50.9921 KOps/s $\color{#35bf28}+0.14\%$
test_step_mdp_speed[False-True-True-True-True] 84.2720μs 52.2512μs 19.1383 KOps/s 18.8351 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[False-True-True-True-False] 89.7610μs 30.9867μs 32.2719 KOps/s 32.0148 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-True-True-False-True] 2.2734ms 33.1846μs 30.1344 KOps/s 30.6224 KOps/s $\color{#d91a1a}-1.59\%$
test_step_mdp_speed[False-True-True-False-False] 57.3810μs 18.6858μs 53.5167 KOps/s 52.1033 KOps/s $\color{#35bf28}+2.71\%$
test_step_mdp_speed[False-True-False-True-True] 89.9120μs 54.2722μs 18.4256 KOps/s 17.6651 KOps/s $\color{#35bf28}+4.31\%$
test_step_mdp_speed[False-True-False-True-False] 60.9910μs 33.8569μs 29.5361 KOps/s 29.3829 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-True-False-False-True] 68.3420μs 34.9671μs 28.5983 KOps/s 28.7503 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[False-True-False-False-False] 55.2510μs 21.3062μs 46.9347 KOps/s 47.0985 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-False-True-True-True] 0.1019ms 56.8409μs 17.5930 KOps/s 17.7621 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-False-True-True-False] 71.7520μs 36.8088μs 27.1674 KOps/s 28.0014 KOps/s $\color{#d91a1a}-2.98\%$
test_step_mdp_speed[False-False-True-False-True] 64.0810μs 35.2658μs 28.3561 KOps/s 29.2148 KOps/s $\color{#d91a1a}-2.94\%$
test_step_mdp_speed[False-False-True-False-False] 88.8620μs 21.1354μs 47.3139 KOps/s 47.3553 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-False-True-True] 96.9410μs 59.1521μs 16.9056 KOps/s 17.2081 KOps/s $\color{#d91a1a}-1.76\%$
test_step_mdp_speed[False-False-False-True-False] 67.5010μs 39.3806μs 25.3932 KOps/s 25.8867 KOps/s $\color{#d91a1a}-1.91\%$
test_step_mdp_speed[False-False-False-False-True] 68.9620μs 37.6081μs 26.5900 KOps/s 27.3846 KOps/s $\color{#d91a1a}-2.90\%$
test_step_mdp_speed[False-False-False-False-False] 57.0510μs 23.8432μs 41.9406 KOps/s 41.9191 KOps/s $\color{#35bf28}+0.05\%$
test_values[generalized_advantage_estimate-True-True] 10.2152ms 9.9263ms 100.7421 Ops/s 101.8038 Ops/s $\color{#d91a1a}-1.04\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.6684ms 17.4731ms 57.2309 Ops/s 57.4175 Ops/s $\color{#d91a1a}-0.32\%$
test_values[td0_return_estimate-False-False] 0.2233ms 0.1277ms 7.8286 KOps/s 7.7786 KOps/s $\color{#35bf28}+0.64\%$
test_values[td1_return_estimate-False-False] 27.3647ms 27.0089ms 37.0248 Ops/s 36.8927 Ops/s $\color{#35bf28}+0.36\%$
test_values[vec_td1_return_estimate-False-False] 17.9104ms 17.5646ms 56.9328 Ops/s 57.0005 Ops/s $\color{#d91a1a}-0.12\%$
test_values[td_lambda_return_estimate-True-False] 40.7366ms 40.3776ms 24.7662 Ops/s 24.7593 Ops/s $\color{#35bf28}+0.03\%$
test_values[vec_td_lambda_return_estimate-True-False] 17.7735ms 17.5079ms 57.1169 Ops/s 57.0873 Ops/s $\color{#35bf28}+0.05\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6655ms 8.5534ms 116.9125 Ops/s 117.4114 Ops/s $\color{#d91a1a}-0.42\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7172ms 1.5188ms 658.3959 Ops/s 665.6461 Ops/s $\color{#d91a1a}-1.09\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5436ms 0.4159ms 2.4046 KOps/s 2.4412 KOps/s $\color{#d91a1a}-1.50\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.6348ms 33.8356ms 29.5547 Ops/s 28.8674 Ops/s $\color{#35bf28}+2.38\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8789ms 1.7139ms 583.4574 Ops/s 586.6630 Ops/s $\color{#d91a1a}-0.55\%$
test_dqn_speed[False-None] 6.4105ms 1.4381ms 695.3654 Ops/s 709.3940 Ops/s $\color{#d91a1a}-1.98\%$
test_dqn_speed[False-backward] 1.9946ms 1.9338ms 517.1139 Ops/s 519.6276 Ops/s $\color{#d91a1a}-0.48\%$
test_dqn_speed[True-None] 0.9147ms 0.5276ms 1.8955 KOps/s 1.9225 KOps/s $\color{#d91a1a}-1.40\%$
test_dqn_speed[True-backward] 1.0023ms 0.9609ms 1.0407 KOps/s 1.0083 KOps/s $\color{#35bf28}+3.21\%$
test_dqn_speed[reduce-overhead-None] 0.7346ms 0.5033ms 1.9870 KOps/s 1.9622 KOps/s $\color{#35bf28}+1.27\%$
test_dqn_speed[reduce-overhead-backward] 1.0862ms 0.9623ms 1.0392 KOps/s 1.0382 KOps/s $\color{#35bf28}+0.10\%$
test_ddpg_speed[False-None] 3.2327ms 2.8795ms 347.2832 Ops/s 340.2974 Ops/s $\color{#35bf28}+2.05\%$
test_ddpg_speed[False-backward] 4.1859ms 4.0877ms 244.6352 Ops/s 238.5351 Ops/s $\color{#35bf28}+2.56\%$
test_ddpg_speed[True-None] 1.7605ms 1.3725ms 728.6101 Ops/s 692.8302 Ops/s $\textbf{\color{#35bf28}+5.16\%}$
test_ddpg_speed[True-backward] 2.4197ms 2.3631ms 423.1663 Ops/s 394.9086 Ops/s $\textbf{\color{#35bf28}+7.16\%}$
test_ddpg_speed[reduce-overhead-None] 1.7588ms 1.3630ms 733.6601 Ops/s 725.6707 Ops/s $\color{#35bf28}+1.10\%$
test_ddpg_speed[reduce-overhead-backward] 2.7757ms 2.3503ms 425.4767 Ops/s 421.7139 Ops/s $\color{#35bf28}+0.89\%$
test_sac_speed[False-None] 8.9226ms 7.9260ms 126.1678 Ops/s 127.3639 Ops/s $\color{#d91a1a}-0.94\%$
test_sac_speed[False-backward] 11.5656ms 11.1654ms 89.5622 Ops/s 90.0479 Ops/s $\color{#d91a1a}-0.54\%$
test_sac_speed[True-None] 2.5090ms 2.1019ms 475.7494 Ops/s 465.0710 Ops/s $\color{#35bf28}+2.30\%$
test_sac_speed[True-backward] 4.9366ms 4.2075ms 237.6684 Ops/s 234.5810 Ops/s $\color{#35bf28}+1.32\%$
test_sac_speed[reduce-overhead-None] 2.2875ms 2.1213ms 471.3996 Ops/s 458.2578 Ops/s $\color{#35bf28}+2.87\%$
test_sac_speed[reduce-overhead-backward] 4.1365ms 3.9813ms 251.1773 Ops/s 240.3853 Ops/s $\color{#35bf28}+4.49\%$
test_redq_speed[False-None] 15.0336ms 10.4017ms 96.1381 Ops/s 94.7553 Ops/s $\color{#35bf28}+1.46\%$
test_redq_speed[False-backward] 18.4919ms 17.7047ms 56.4822 Ops/s 55.8735 Ops/s $\color{#35bf28}+1.09\%$
test_redq_speed[True-None] 4.8700ms 4.3817ms 228.2225 Ops/s 192.8441 Ops/s $\textbf{\color{#35bf28}+18.35\%}$
test_redq_speed[True-backward] 9.9965ms 9.7753ms 102.2987 Ops/s 96.1509 Ops/s $\textbf{\color{#35bf28}+6.39\%}$
test_redq_speed[reduce-overhead-None] 4.9682ms 4.3628ms 229.2093 Ops/s 219.2717 Ops/s $\color{#35bf28}+4.53\%$
test_redq_speed[reduce-overhead-backward] 10.1189ms 9.9259ms 100.7470 Ops/s 102.8871 Ops/s $\color{#d91a1a}-2.08\%$
test_redq_deprec_speed[False-None] 11.4781ms 10.9199ms 91.5762 Ops/s 90.9649 Ops/s $\color{#35bf28}+0.67\%$
test_redq_deprec_speed[False-backward] 16.2042ms 15.7371ms 63.5441 Ops/s 63.6700 Ops/s $\color{#d91a1a}-0.20\%$
test_redq_deprec_speed[True-None] 4.1163ms 3.6020ms 277.6265 Ops/s 273.9699 Ops/s $\color{#35bf28}+1.33\%$
test_redq_deprec_speed[True-backward] 7.7446ms 7.4221ms 134.7324 Ops/s 135.9831 Ops/s $\color{#d91a1a}-0.92\%$
test_redq_deprec_speed[reduce-overhead-None] 3.8786ms 3.6050ms 277.3949 Ops/s 283.4951 Ops/s $\color{#d91a1a}-2.15\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.6508ms 7.4323ms 134.5484 Ops/s 138.8090 Ops/s $\color{#d91a1a}-3.07\%$
test_td3_speed[False-None] 8.3329ms 8.0028ms 124.9556 Ops/s 119.9757 Ops/s $\color{#35bf28}+4.15\%$
test_td3_speed[False-backward] 11.6097ms 10.9471ms 91.3481 Ops/s 92.3012 Ops/s $\color{#d91a1a}-1.03\%$
test_td3_speed[True-None] 1.8324ms 1.7721ms 564.3106 Ops/s 548.2588 Ops/s $\color{#35bf28}+2.93\%$
test_td3_speed[True-backward] 3.7060ms 3.5684ms 280.2402 Ops/s 273.0733 Ops/s $\color{#35bf28}+2.62\%$
test_td3_speed[reduce-overhead-None] 1.8213ms 1.7576ms 568.9688 Ops/s 550.5387 Ops/s $\color{#35bf28}+3.35\%$
test_td3_speed[reduce-overhead-backward] 3.7033ms 3.6067ms 277.2604 Ops/s 271.7652 Ops/s $\color{#35bf28}+2.02\%$
test_cql_speed[False-None] 29.1027ms 25.9774ms 38.4951 Ops/s 38.4629 Ops/s $\color{#35bf28}+0.08\%$
test_cql_speed[False-backward] 38.8473ms 35.4272ms 28.2269 Ops/s 28.3801 Ops/s $\color{#d91a1a}-0.54\%$
test_cql_speed[True-None] 12.6932ms 12.1884ms 82.0452 Ops/s 80.5421 Ops/s $\color{#35bf28}+1.87\%$
test_cql_speed[True-backward] 18.9492ms 18.1969ms 54.9544 Ops/s 52.6131 Ops/s $\color{#35bf28}+4.45\%$
test_cql_speed[reduce-overhead-None] 12.8748ms 12.3366ms 81.0593 Ops/s 79.5237 Ops/s $\color{#35bf28}+1.93\%$
test_cql_speed[reduce-overhead-backward] 19.0359ms 18.2234ms 54.8745 Ops/s 54.7720 Ops/s $\color{#35bf28}+0.19\%$
test_a2c_speed[False-None] 5.9617ms 5.4066ms 184.9596 Ops/s 183.7865 Ops/s $\color{#35bf28}+0.64\%$
test_a2c_speed[False-backward] 12.2885ms 11.8833ms 84.1516 Ops/s 84.2139 Ops/s $\color{#d91a1a}-0.07\%$
test_a2c_speed[True-None] 3.8232ms 3.6977ms 270.4403 Ops/s 269.5767 Ops/s $\color{#35bf28}+0.32\%$
test_a2c_speed[True-backward] 8.7019ms 8.4988ms 117.6640 Ops/s 110.9089 Ops/s $\textbf{\color{#35bf28}+6.09\%}$
test_a2c_speed[reduce-overhead-None] 4.1024ms 3.7133ms 269.3021 Ops/s 271.6623 Ops/s $\color{#d91a1a}-0.87\%$
test_a2c_speed[reduce-overhead-backward] 9.1622ms 8.8356ms 113.1791 Ops/s 113.3204 Ops/s $\color{#d91a1a}-0.12\%$
test_ppo_speed[False-None] 6.2805ms 5.9095ms 169.2180 Ops/s 171.6761 Ops/s $\color{#d91a1a}-1.43\%$
test_ppo_speed[False-backward] 12.9382ms 12.5438ms 79.7204 Ops/s 80.3247 Ops/s $\color{#d91a1a}-0.75\%$
test_ppo_speed[True-None] 4.0558ms 3.6475ms 274.1616 Ops/s 275.3795 Ops/s $\color{#d91a1a}-0.44\%$
test_ppo_speed[True-backward] 8.5983ms 8.4300ms 118.6246 Ops/s 115.6321 Ops/s $\color{#35bf28}+2.59\%$
test_ppo_speed[reduce-overhead-None] 3.8024ms 3.6008ms 277.7141 Ops/s 274.6830 Ops/s $\color{#35bf28}+1.10\%$
test_ppo_speed[reduce-overhead-backward] 9.2658ms 8.7190ms 114.6926 Ops/s 114.1855 Ops/s $\color{#35bf28}+0.44\%$
test_reinforce_speed[False-None] 4.9942ms 4.6080ms 217.0162 Ops/s 215.9981 Ops/s $\color{#35bf28}+0.47\%$
test_reinforce_speed[False-backward] 7.7435ms 7.4591ms 134.0652 Ops/s 134.4556 Ops/s $\color{#d91a1a}-0.29\%$
test_reinforce_speed[True-None] 3.2128ms 2.8451ms 351.4865 Ops/s 346.4469 Ops/s $\color{#35bf28}+1.45\%$
test_reinforce_speed[True-backward] 8.3519ms 7.7242ms 129.4628 Ops/s 129.3313 Ops/s $\color{#35bf28}+0.10\%$
test_reinforce_speed[reduce-overhead-None] 3.3859ms 2.8436ms 351.6613 Ops/s 351.2145 Ops/s $\color{#35bf28}+0.13\%$
test_reinforce_speed[reduce-overhead-backward] 8.0998ms 7.8461ms 127.4523 Ops/s 124.4316 Ops/s $\color{#35bf28}+2.43\%$
test_iql_speed[False-None] 22.8064ms 20.1326ms 49.6706 Ops/s 50.3439 Ops/s $\color{#d91a1a}-1.34\%$
test_iql_speed[False-backward] 35.7767ms 30.6973ms 32.5761 Ops/s 32.9556 Ops/s $\color{#d91a1a}-1.15\%$
test_iql_speed[True-None] 8.7641ms 8.4472ms 118.3819 Ops/s 111.0499 Ops/s $\textbf{\color{#35bf28}+6.60\%}$
test_iql_speed[True-backward] 16.9254ms 16.5574ms 60.3958 Ops/s 60.1023 Ops/s $\color{#35bf28}+0.49\%$
test_iql_speed[reduce-overhead-None] 8.8704ms 8.4929ms 117.7451 Ops/s 115.7760 Ops/s $\color{#35bf28}+1.70\%$
test_iql_speed[reduce-overhead-backward] 17.3035ms 16.8908ms 59.2038 Ops/s 58.8279 Ops/s $\color{#35bf28}+0.64\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5362ms 6.0469ms 165.3731 Ops/s 166.4241 Ops/s $\color{#d91a1a}-0.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5379ms 0.2943ms 3.3981 KOps/s 3.1264 KOps/s $\textbf{\color{#35bf28}+8.69\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8722ms 0.2991ms 3.3434 KOps/s 3.8372 KOps/s $\textbf{\color{#d91a1a}-12.87\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4480ms 5.7627ms 173.5305 Ops/s 175.1250 Ops/s $\color{#d91a1a}-0.91\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5841s 0.8234ms 1.2145 KOps/s 3.4868 KOps/s $\textbf{\color{#d91a1a}-65.17\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5494ms 0.3511ms 2.8484 KOps/s 3.3312 KOps/s $\textbf{\color{#d91a1a}-14.49\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.9239ms 1.4349ms 696.9076 Ops/s 803.9459 Ops/s $\textbf{\color{#d91a1a}-13.31\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.8130ms 1.3425ms 744.8797 Ops/s 863.5853 Ops/s $\textbf{\color{#d91a1a}-13.75\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2836ms 5.9021ms 169.4317 Ops/s 170.6106 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7786ms 0.4469ms 2.2378 KOps/s 2.0789 KOps/s $\textbf{\color{#35bf28}+7.64\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6635ms 0.4415ms 2.2653 KOps/s 2.3032 KOps/s $\color{#d91a1a}-1.65\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1667ms 5.7668ms 173.4079 Ops/s 175.9040 Ops/s $\color{#d91a1a}-1.42\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.1810ms 0.3546ms 2.8197 KOps/s 819.3858 Ops/s $\textbf{\color{#35bf28}+244.12\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5306ms 0.3399ms 2.9424 KOps/s 3.0803 KOps/s $\color{#d91a1a}-4.48\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2430ms 5.7065ms 175.2376 Ops/s 173.9654 Ops/s $\color{#35bf28}+0.73\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8689ms 0.3183ms 3.1415 KOps/s 3.0997 KOps/s $\color{#35bf28}+1.35\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5252ms 0.3077ms 3.2498 KOps/s 3.1327 KOps/s $\color{#35bf28}+3.74\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.9607ms 6.0384ms 165.6060 Ops/s 168.3412 Ops/s $\color{#d91a1a}-1.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7146ms 0.4468ms 2.2383 KOps/s 1.9597 KOps/s $\textbf{\color{#35bf28}+14.21\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6315ms 0.4196ms 2.3832 KOps/s 2.0952 KOps/s $\textbf{\color{#35bf28}+13.75\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.5146s 15.2439ms 65.5999 Ops/s 196.2223 Ops/s $\textbf{\color{#d91a1a}-66.57\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.8842ms 1.7360ms 576.0238 Ops/s 428.8776 Ops/s $\textbf{\color{#35bf28}+34.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 9.2612ms 1.1746ms 851.3201 Ops/s 827.1973 Ops/s $\color{#35bf28}+2.92\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.0620ms 5.0618ms 197.5570 Ops/s 54.7150 Ops/s $\textbf{\color{#35bf28}+261.07\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.0476ms 2.0530ms 487.0922 Ops/s 508.7522 Ops/s $\color{#d91a1a}-4.26\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.6087ms 1.2442ms 803.7000 Ops/s 806.6001 Ops/s $\color{#d91a1a}-0.36\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5132s 15.4784ms 64.6060 Ops/s 186.3828 Ops/s $\textbf{\color{#d91a1a}-65.34\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.0626ms 2.1749ms 459.7835 Ops/s 438.0443 Ops/s $\color{#35bf28}+4.96\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.3589ms 1.0187ms 981.6752 Ops/s 714.3278 Ops/s $\textbf{\color{#35bf28}+37.43\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.9304ms 32.8534ms 30.4382 Ops/s 29.5360 Ops/s $\color{#35bf28}+3.05\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.5251ms 17.4388ms 57.3433 Ops/s 56.5852 Ops/s $\color{#35bf28}+1.34\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 35.5296ms 33.5823ms 29.7776 Ops/s 29.1403 Ops/s $\color{#35bf28}+2.19\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.3437ms 17.8122ms 56.1412 Ops/s 56.7532 Ops/s $\color{#d91a1a}-1.08\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 36.9278ms 35.4520ms 28.2071 Ops/s 27.5811 Ops/s $\color{#35bf28}+2.27\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.1051ms 19.1325ms 52.2670 Ops/s 51.2234 Ops/s $\color{#35bf28}+2.04\%$

vmoens added 7 commits October 19, 2025 20:56
@vmoens
Update
39f3850 
[ghstack-poisoned]
@vmoens
Update
70d35e9 
[ghstack-poisoned]
@vmoens
Update
72d9116 
[ghstack-poisoned]
@vmoens
Update
dfaa0fa 
[ghstack-poisoned]
@vmoens
Update
264938d 
[ghstack-poisoned]
@vmoens
Update
f2901e8 
[ghstack-poisoned]
@vmoens
Update
4051ef3 
[ghstack-poisoned]
This was referenced Oct 22, 2025
Merged
Open
vmoens added 11 commits October 22, 2025 12:31
@vmoens
Update
cb20e00 
[ghstack-poisoned]
@vmoens
Update
70768b5 
[ghstack-poisoned]
@vmoens
Update
b7b0605 
[ghstack-poisoned]
@vmoens
Update
d7a5775 
[ghstack-poisoned]
@vmoens
Update
1aca445 
[ghstack-poisoned]
@vmoens
Update
f68fbfa 
[ghstack-poisoned]
@vmoens
Update
6ad5bc6 
[ghstack-poisoned]
@vmoens
Update
6c07b5e 
[ghstack-poisoned]
@vmoens
Update
cc3994b 
[ghstack-poisoned]
@vmoens
Update
9259bad 
[ghstack-poisoned]
@vmoens
Update
6f4fc6d 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens