Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Feature] Trainer Algorithms - Configuration System #3189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 29 commits into
base: gh/vmoens/152/base
Choose a base branch
from
Open

[Feature] Trainer Algorithms - Configuration System #3189

wants to merge 29 commits into from

Conversation

vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 14, 2025 •
edited
Loading

@vmoens
Update
40c8b63 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 14, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3189

Note: Links to docs will display an error until the docs builds have been completed.

❌ 11 New Failures, 1 Unrelated Failure

As of commit ffd35d7 with merge base 963fdd4 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 14, 2025
Merged
Merged
Merged
Merged
Merged
Open
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 14, 2025
vmoens added 7 commits October 16, 2025 15:00
@vmoens
Update
74bfb5a 
[ghstack-poisoned]
@vmoens
Update
bfd08f5 
[ghstack-poisoned]
@vmoens
Update
eb85308 
[ghstack-poisoned]
@vmoens
Update
0d1f7ce 
[ghstack-poisoned]
@vmoens
Update
351326c 
[ghstack-poisoned]
@vmoens
Update
bb95ddb 
[ghstack-poisoned]
@vmoens
Update
5fcf578 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 18, 2025
Open
vmoens added 4 commits October 18, 2025 19:00
@vmoens
Update
67659d3 
[ghstack-poisoned]
@vmoens
Update
1f9b4c8 
[ghstack-poisoned]
@vmoens
Update
36e3097 
[ghstack-poisoned]
@vmoens
Update
5b3b689 
[ghstack-poisoned]
@vmoens vmoens added the enhancement New feature or request label Oct 20, 2025
@vmoens
Update
ddb96c1 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
920f08a 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Oct 20, 2025
Merged
@vmoens
Update
e5e892f 
[ghstack-poisoned]
@github-actions
Copy link

github-actions bot commented Oct 20, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}39$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 85.6991μs 83.9322μs 11.9144 KOps/s 11.9648 KOps/s $\color{#d91a1a}-0.42\%$
test_tensor_to_bytestream_speed[torch.save] 0.1408ms 0.1400ms 7.1453 KOps/s 6.9517 KOps/s $\color{#35bf28}+2.78\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1225s 0.1221s 8.1910 Ops/s 8.0008 Ops/s $\color{#35bf28}+2.38\%$
test_tensor_to_bytestream_speed[numpy] 2.7883μs 2.7823μs 359.4159 KOps/s 358.3081 KOps/s $\color{#35bf28}+0.31\%$
test_tensor_to_bytestream_speed[safetensors] 43.4376μs 43.2390μs 23.1273 KOps/s 23.3750 KOps/s $\color{#d91a1a}-1.06\%$
test_simple 0.5685s 0.5652s 1.7693 Ops/s 1.7189 Ops/s $\color{#35bf28}+2.93\%$
test_transformed 1.2585s 1.1540s 0.8666 Ops/s 0.8683 Ops/s $\color{#d91a1a}-0.20\%$
test_serial 1.6792s 1.6602s 0.6023 Ops/s 0.5817 Ops/s $\color{#35bf28}+3.54\%$
test_parallel 1.1828s 1.1070s 0.9034 Ops/s 0.8752 Ops/s $\color{#35bf28}+3.22\%$
test_step_mdp_speed[True-True-True-True-True] 0.1463ms 45.4255μs 22.0140 KOps/s 22.6787 KOps/s $\color{#d91a1a}-2.93\%$
test_step_mdp_speed[True-True-True-True-False] 58.2710μs 25.5810μs 39.0915 KOps/s 39.2678 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-True-True-False-True] 63.3510μs 25.6779μs 38.9440 KOps/s 39.3738 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-True-False-False] 43.3010μs 13.9938μs 71.4604 KOps/s 72.0592 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-True-False-True-True] 92.4920μs 48.3589μs 20.6787 KOps/s 20.7081 KOps/s $\color{#d91a1a}-0.14\%$
test_step_mdp_speed[True-True-False-True-False] 65.3710μs 28.1751μs 35.4923 KOps/s 35.7083 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[True-True-False-False-True] 76.7620μs 28.9354μs 34.5598 KOps/s 35.7555 KOps/s $\color{#d91a1a}-3.34\%$
test_step_mdp_speed[True-True-False-False-False] 74.0220μs 16.9150μs 59.1190 KOps/s 61.0141 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[True-False-True-True-True] 0.1140ms 51.3591μs 19.4707 KOps/s 20.0420 KOps/s $\color{#d91a1a}-2.85\%$
test_step_mdp_speed[True-False-True-True-False] 96.4220μs 30.7596μs 32.5101 KOps/s 32.6192 KOps/s $\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-False-True-False-True] 72.1920μs 28.4976μs 35.0907 KOps/s 35.1792 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-False-True-False-False] 44.5810μs 17.0721μs 58.5751 KOps/s 59.6213 KOps/s $\color{#d91a1a}-1.75\%$
test_step_mdp_speed[True-False-False-True-True] 99.3020μs 54.6084μs 18.3122 KOps/s 18.5893 KOps/s $\color{#d91a1a}-1.49\%$
test_step_mdp_speed[True-False-False-True-False] 0.1205ms 33.6203μs 29.7439 KOps/s 30.0802 KOps/s $\color{#d91a1a}-1.12\%$
test_step_mdp_speed[True-False-False-False-True] 71.0810μs 31.5931μs 31.6525 KOps/s 33.0957 KOps/s $\color{#d91a1a}-4.36\%$
test_step_mdp_speed[True-False-False-False-False] 45.0510μs 19.7486μs 50.6364 KOps/s 51.4913 KOps/s $\color{#d91a1a}-1.66\%$
test_step_mdp_speed[False-True-True-True-True] 92.3120μs 52.2375μs 19.1433 KOps/s 19.5888 KOps/s $\color{#d91a1a}-2.27\%$
test_step_mdp_speed[False-True-True-True-False] 0.1030ms 31.4669μs 31.7795 KOps/s 32.6844 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-True-True-False-True] 2.3760ms 33.4018μs 29.9385 KOps/s 31.0826 KOps/s $\color{#d91a1a}-3.68\%$
test_step_mdp_speed[False-True-True-False-False] 48.8010μs 19.3062μs 51.7968 KOps/s 53.4009 KOps/s $\color{#d91a1a}-3.00\%$
test_step_mdp_speed[False-True-False-True-True] 91.1420μs 55.0394μs 18.1688 KOps/s 18.7015 KOps/s $\color{#d91a1a}-2.85\%$
test_step_mdp_speed[False-True-False-True-False] 68.8810μs 34.4491μs 29.0284 KOps/s 29.4263 KOps/s $\color{#d91a1a}-1.35\%$
test_step_mdp_speed[False-True-False-False-True] 70.2920μs 36.1069μs 27.6955 KOps/s 29.3330 KOps/s $\textbf{\color{#d91a1a}-5.58\%}$
test_step_mdp_speed[False-True-False-False-False] 80.7720μs 22.1485μs 45.1498 KOps/s 47.1419 KOps/s $\color{#d91a1a}-4.23\%$
test_step_mdp_speed[False-False-True-True-True] 0.1180ms 57.5919μs 17.3636 KOps/s 17.8192 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-False-True-True-False] 70.6110μs 37.2736μs 26.8287 KOps/s 27.3405 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[False-False-True-False-True] 96.6420μs 35.6360μs 28.0615 KOps/s 29.1409 KOps/s $\color{#d91a1a}-3.70\%$
test_step_mdp_speed[False-False-True-False-False] 63.2710μs 21.7751μs 45.9240 KOps/s 47.1160 KOps/s $\color{#d91a1a}-2.53\%$
test_step_mdp_speed[False-False-False-True-True] 87.2410μs 59.6836μs 16.7550 KOps/s 17.0422 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-False-False-True-False] 0.1178ms 39.5986μs 25.2534 KOps/s 25.7900 KOps/s $\color{#d91a1a}-2.08\%$
test_step_mdp_speed[False-False-False-False-True] 72.4220μs 38.3685μs 26.0631 KOps/s 27.2554 KOps/s $\color{#d91a1a}-4.37\%$
test_step_mdp_speed[False-False-False-False-False] 80.8620μs 24.4767μs 40.8552 KOps/s 41.7787 KOps/s $\color{#d91a1a}-2.21\%$
test_values[generalized_advantage_estimate-True-True] 11.3956ms 10.1959ms 98.0785 Ops/s 93.8651 Ops/s $\color{#35bf28}+4.49\%$
test_values[vec_generalized_advantage_estimate-True-True] 17.5682ms 11.2977ms 88.5139 Ops/s 57.3088 Ops/s $\textbf{\color{#35bf28}+54.45\%}$
test_values[td0_return_estimate-False-False] 0.2370ms 0.1286ms 7.7773 KOps/s 7.6674 KOps/s $\color{#35bf28}+1.43\%$
test_values[td1_return_estimate-False-False] 28.4181ms 27.8474ms 35.9100 Ops/s 34.3025 Ops/s $\color{#35bf28}+4.69\%$
test_values[vec_td1_return_estimate-False-False] 11.3073ms 11.1399ms 89.7676 Ops/s 57.0258 Ops/s $\textbf{\color{#35bf28}+57.42\%}$
test_values[td_lambda_return_estimate-True-False] 42.2152ms 41.8374ms 23.9020 Ops/s 23.1043 Ops/s $\color{#35bf28}+3.45\%$
test_values[vec_td_lambda_return_estimate-True-False] 12.0749ms 11.1526ms 89.6653 Ops/s 57.1689 Ops/s $\textbf{\color{#35bf28}+56.84\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9850ms 8.8535ms 112.9499 Ops/s 108.3257 Ops/s $\color{#35bf28}+4.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7475ms 1.5143ms 660.3762 Ops/s 659.6617 Ops/s $\color{#35bf28}+0.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4697ms 0.4155ms 2.4070 KOps/s 2.3720 KOps/s $\color{#35bf28}+1.47\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.5241ms 31.4604ms 31.7860 Ops/s 28.7511 Ops/s $\textbf{\color{#35bf28}+10.56\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.0999ms 1.7134ms 583.6456 Ops/s 590.2018 Ops/s $\color{#d91a1a}-1.11\%$
test_dqn_speed[False-None] 1.5592ms 1.4303ms 699.1449 Ops/s 667.3189 Ops/s $\color{#35bf28}+4.77\%$
test_dqn_speed[False-backward] 2.0050ms 1.9428ms 514.7234 Ops/s 512.4729 Ops/s $\color{#35bf28}+0.44\%$
test_dqn_speed[True-None] 0.6635ms 0.5124ms 1.9518 KOps/s 1.8862 KOps/s $\color{#35bf28}+3.47\%$
test_dqn_speed[True-backward] 1.0155ms 0.9671ms 1.0340 KOps/s 928.4342 Ops/s $\textbf{\color{#35bf28}+11.37\%}$
test_dqn_speed[reduce-overhead-None] 0.5650ms 0.5049ms 1.9805 KOps/s 1.8955 KOps/s $\color{#35bf28}+4.48\%$
test_dqn_speed[reduce-overhead-backward] 0.9985ms 0.9605ms 1.0411 KOps/s 1.0264 KOps/s $\color{#35bf28}+1.43\%$
test_ddpg_speed[False-None] 3.2241ms 2.8842ms 346.7108 Ops/s 340.2831 Ops/s $\color{#35bf28}+1.89\%$
test_ddpg_speed[False-backward] 4.2077ms 4.1058ms 243.5561 Ops/s 237.0825 Ops/s $\color{#35bf28}+2.73\%$
test_ddpg_speed[True-None] 1.5085ms 1.3862ms 721.3967 Ops/s 703.2498 Ops/s $\color{#35bf28}+2.58\%$
test_ddpg_speed[True-backward] 2.4520ms 2.3831ms 419.6165 Ops/s 346.0682 Ops/s $\textbf{\color{#35bf28}+21.25\%}$
test_ddpg_speed[reduce-overhead-None] 1.5200ms 1.3817ms 723.7415 Ops/s 687.0436 Ops/s $\textbf{\color{#35bf28}+5.34\%}$
test_ddpg_speed[reduce-overhead-backward] 2.4516ms 2.3647ms 422.8807 Ops/s 418.2562 Ops/s $\color{#35bf28}+1.11\%$
test_sac_speed[False-None] 8.3790ms 7.9171ms 126.3093 Ops/s 125.4107 Ops/s $\color{#35bf28}+0.72\%$
test_sac_speed[False-backward] 11.5320ms 11.2198ms 89.1281 Ops/s 88.9133 Ops/s $\color{#35bf28}+0.24\%$
test_sac_speed[True-None] 2.2994ms 2.1145ms 472.9226 Ops/s 468.1122 Ops/s $\color{#35bf28}+1.03\%$
test_sac_speed[True-backward] 4.3399ms 4.0581ms 246.4213 Ops/s 208.3431 Ops/s $\textbf{\color{#35bf28}+18.28\%}$
test_sac_speed[reduce-overhead-None] 2.2886ms 2.1049ms 475.0908 Ops/s 440.4364 Ops/s $\textbf{\color{#35bf28}+7.87\%}$
test_sac_speed[reduce-overhead-backward] 4.1637ms 4.0731ms 245.5105 Ops/s 233.2857 Ops/s $\textbf{\color{#35bf28}+5.24\%}$
test_redq_speed[False-None] 10.7653ms 10.3708ms 96.4245 Ops/s 93.7105 Ops/s $\color{#35bf28}+2.90\%$
test_redq_speed[False-backward] 18.6358ms 17.9638ms 55.6674 Ops/s 50.7141 Ops/s $\textbf{\color{#35bf28}+9.77\%}$
test_redq_speed[True-None] 4.5942ms 4.3558ms 229.5807 Ops/s 217.8192 Ops/s $\textbf{\color{#35bf28}+5.40\%}$
test_redq_speed[True-backward] 10.1080ms 9.8529ms 101.4928 Ops/s 98.6822 Ops/s $\color{#35bf28}+2.85\%$
test_redq_speed[reduce-overhead-None] 4.6352ms 4.3976ms 227.3944 Ops/s 223.7950 Ops/s $\color{#35bf28}+1.61\%$
test_redq_speed[reduce-overhead-backward] 10.6251ms 10.1185ms 98.8290 Ops/s 100.2250 Ops/s $\color{#d91a1a}-1.39\%$
test_redq_deprec_speed[False-None] 11.5059ms 11.0484ms 90.5110 Ops/s 91.1853 Ops/s $\color{#d91a1a}-0.74\%$
test_redq_deprec_speed[False-backward] 16.4470ms 16.0344ms 62.3658 Ops/s 62.1423 Ops/s $\color{#35bf28}+0.36\%$
test_redq_deprec_speed[True-None] 3.7499ms 3.6078ms 277.1801 Ops/s 260.4442 Ops/s $\textbf{\color{#35bf28}+6.43\%}$
test_redq_deprec_speed[True-backward] 7.9032ms 7.6371ms 130.9395 Ops/s 120.4460 Ops/s $\textbf{\color{#35bf28}+8.71\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.8286ms 3.5943ms 278.2192 Ops/s 276.3086 Ops/s $\color{#35bf28}+0.69\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.8934ms 7.6076ms 131.4474 Ops/s 126.4240 Ops/s $\color{#35bf28}+3.97\%$
test_td3_speed[False-None] 8.9641ms 8.0207ms 124.6775 Ops/s 124.5362 Ops/s $\color{#35bf28}+0.11\%$
test_td3_speed[False-backward] 11.5438ms 10.9677ms 91.1765 Ops/s 91.2434 Ops/s $\color{#d91a1a}-0.07\%$
test_td3_speed[True-None] 1.8378ms 1.8037ms 554.4254 Ops/s 547.2326 Ops/s $\color{#35bf28}+1.31\%$
test_td3_speed[True-backward] 3.7355ms 3.6151ms 276.6199 Ops/s 237.4024 Ops/s $\textbf{\color{#35bf28}+16.52\%}$
test_td3_speed[reduce-overhead-None] 1.7963ms 1.7731ms 563.9725 Ops/s 552.8100 Ops/s $\color{#35bf28}+2.02\%$
test_td3_speed[reduce-overhead-backward] 3.7799ms 3.6894ms 271.0497 Ops/s 252.0834 Ops/s $\textbf{\color{#35bf28}+7.52\%}$
test_cql_speed[False-None] 28.6821ms 26.1494ms 38.2419 Ops/s 38.2155 Ops/s $\color{#35bf28}+0.07\%$
test_cql_speed[False-backward] 36.1160ms 35.2489ms 28.3697 Ops/s 27.9445 Ops/s $\color{#35bf28}+1.52\%$
test_cql_speed[True-None] 12.7503ms 12.4059ms 80.6066 Ops/s 81.6756 Ops/s $\color{#d91a1a}-1.31\%$
test_cql_speed[True-backward] 18.8460ms 18.5129ms 54.0165 Ops/s 54.8142 Ops/s $\color{#d91a1a}-1.46\%$
test_cql_speed[reduce-overhead-None] 12.9568ms 12.5376ms 79.7600 Ops/s 78.9057 Ops/s $\color{#35bf28}+1.08\%$
test_cql_speed[reduce-overhead-backward] 18.6647ms 18.4051ms 54.3326 Ops/s 54.8220 Ops/s $\color{#d91a1a}-0.89\%$
test_a2c_speed[False-None] 6.1401ms 5.4269ms 184.2689 Ops/s 178.8262 Ops/s $\color{#35bf28}+3.04\%$
test_a2c_speed[False-backward] 12.2963ms 12.0466ms 83.0113 Ops/s 82.8845 Ops/s $\color{#35bf28}+0.15\%$
test_a2c_speed[True-None] 3.9139ms 3.7244ms 268.4964 Ops/s 254.1099 Ops/s $\textbf{\color{#35bf28}+5.66\%}$
test_a2c_speed[True-backward] 9.0351ms 8.6096ms 116.1488 Ops/s 108.2122 Ops/s $\textbf{\color{#35bf28}+7.33\%}$
test_a2c_speed[reduce-overhead-None] 4.1114ms 3.7238ms 268.5396 Ops/s 263.8838 Ops/s $\color{#35bf28}+1.76\%$
test_a2c_speed[reduce-overhead-backward] 9.0697ms 8.8774ms 112.6459 Ops/s 104.6117 Ops/s $\textbf{\color{#35bf28}+7.68\%}$
test_ppo_speed[False-None] 6.2529ms 6.0009ms 166.6420 Ops/s 164.3674 Ops/s $\color{#35bf28}+1.38\%$
test_ppo_speed[False-backward] 13.0327ms 12.7597ms 78.3717 Ops/s 77.8619 Ops/s $\color{#35bf28}+0.65\%$
test_ppo_speed[True-None] 3.7899ms 3.6500ms 273.9758 Ops/s 266.7154 Ops/s $\color{#35bf28}+2.72\%$
test_ppo_speed[True-backward] 8.7269ms 8.5350ms 117.1644 Ops/s 116.1365 Ops/s $\color{#35bf28}+0.89\%$
test_ppo_speed[reduce-overhead-None] 4.1662ms 3.6238ms 275.9569 Ops/s 268.8881 Ops/s $\color{#35bf28}+2.63\%$
test_ppo_speed[reduce-overhead-backward] 8.9930ms 8.7800ms 113.8954 Ops/s 111.4150 Ops/s $\color{#35bf28}+2.23\%$
test_reinforce_speed[False-None] 5.0281ms 4.6401ms 215.5109 Ops/s 209.5761 Ops/s $\color{#35bf28}+2.83\%$
test_reinforce_speed[False-backward] 7.6976ms 7.4656ms 133.9475 Ops/s 131.0619 Ops/s $\color{#35bf28}+2.20\%$
test_reinforce_speed[True-None] 3.0773ms 2.8810ms 347.1065 Ops/s 332.3782 Ops/s $\color{#35bf28}+4.43\%$
test_reinforce_speed[True-backward] 7.9879ms 7.7455ms 129.1065 Ops/s 121.1016 Ops/s $\textbf{\color{#35bf28}+6.61\%}$
test_reinforce_speed[reduce-overhead-None] 3.0215ms 2.8620ms 349.4094 Ops/s 334.9400 Ops/s $\color{#35bf28}+4.32\%$
test_reinforce_speed[reduce-overhead-backward] 8.2459ms 7.8848ms 126.8258 Ops/s 125.3382 Ops/s $\color{#35bf28}+1.19\%$
test_iql_speed[False-None] 24.5457ms 20.2894ms 49.2867 Ops/s 50.1554 Ops/s $\color{#d91a1a}-1.73\%$
test_iql_speed[False-backward] 35.2706ms 30.7281ms 32.5435 Ops/s 32.8006 Ops/s $\color{#d91a1a}-0.78\%$
test_iql_speed[True-None] 8.7863ms 8.5057ms 117.5688 Ops/s 115.0711 Ops/s $\color{#35bf28}+2.17\%$
test_iql_speed[True-backward] 17.2192ms 16.8426ms 59.3733 Ops/s 55.7872 Ops/s $\textbf{\color{#35bf28}+6.43\%}$
test_iql_speed[reduce-overhead-None] 8.8630ms 8.5365ms 117.1437 Ops/s 117.2373 Ops/s $\color{#d91a1a}-0.08\%$
test_iql_speed[reduce-overhead-backward] 19.2187ms 17.3955ms 57.4862 Ops/s 57.4068 Ops/s $\color{#35bf28}+0.14\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5508ms 6.0590ms 165.0433 Ops/s 166.9151 Ops/s $\color{#d91a1a}-1.12\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6037ms 0.3058ms 3.2701 KOps/s 2.9084 KOps/s $\textbf{\color{#35bf28}+12.44\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6024ms 0.2901ms 3.4473 KOps/s 3.1888 KOps/s $\textbf{\color{#35bf28}+8.11\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9706ms 5.7616ms 173.5624 Ops/s 175.7778 Ops/s $\color{#d91a1a}-1.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.0721ms 0.2818ms 3.5488 KOps/s 2.9607 KOps/s $\textbf{\color{#35bf28}+19.86\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4863ms 0.2551ms 3.9196 KOps/s 3.0796 KOps/s $\textbf{\color{#35bf28}+27.28\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4449ms 1.2530ms 798.1086 Ops/s 681.5022 Ops/s $\textbf{\color{#35bf28}+17.11\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3545ms 1.1698ms 854.8470 Ops/s 744.4645 Ops/s $\textbf{\color{#35bf28}+14.83\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 10.0996ms 6.0663ms 164.8453 Ops/s 169.9134 Ops/s $\color{#d91a1a}-2.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9702ms 0.4232ms 2.3629 KOps/s 2.1473 KOps/s $\textbf{\color{#35bf28}+10.04\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6491ms 0.4057ms 2.4651 KOps/s 2.2000 KOps/s $\textbf{\color{#35bf28}+12.05\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8424ms 5.7374ms 174.2941 Ops/s 174.0028 Ops/s $\color{#35bf28}+0.17\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0676ms 0.2884ms 3.4676 KOps/s 848.0063 Ops/s $\textbf{\color{#35bf28}+308.91\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5446ms 0.2900ms 3.4482 KOps/s 3.3578 KOps/s $\color{#35bf28}+2.69\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9119ms 5.7037ms 175.3260 Ops/s 172.2294 Ops/s $\color{#35bf28}+1.80\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9781ms 0.2739ms 3.6506 KOps/s 3.1356 KOps/s $\textbf{\color{#35bf28}+16.42\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4700ms 0.2541ms 3.9355 KOps/s 3.5521 KOps/s $\textbf{\color{#35bf28}+10.79\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4179ms 5.8638ms 170.5380 Ops/s 167.1430 Ops/s $\color{#35bf28}+2.03\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8410ms 0.5076ms 1.9699 KOps/s 1.9363 KOps/s $\color{#35bf28}+1.74\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6990ms 0.4418ms 2.2633 KOps/s 2.1379 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.5325s 15.6052ms 64.0812 Ops/s 189.7480 Ops/s $\textbf{\color{#d91a1a}-66.23\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 10.1695ms 2.0436ms 489.3334 Ops/s 419.5573 Ops/s $\textbf{\color{#35bf28}+16.63\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.4508ms 1.1601ms 862.0219 Ops/s 927.2000 Ops/s $\textbf{\color{#d91a1a}-7.03\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.6358ms 5.0875ms 196.5597 Ops/s 55.7324 Ops/s $\textbf{\color{#35bf28}+252.68\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.6142ms 2.0532ms 487.0420 Ops/s 688.2178 Ops/s $\textbf{\color{#d91a1a}-29.23\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.3060ms 1.0924ms 915.4167 Ops/s 780.2751 Ops/s $\textbf{\color{#35bf28}+17.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4980s 15.1654ms 65.9394 Ops/s 181.1052 Ops/s $\textbf{\color{#d91a1a}-63.59\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.3787ms 2.1186ms 472.0164 Ops/s 445.0757 Ops/s $\textbf{\color{#35bf28}+6.05\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 8.6543ms 1.3049ms 766.3598 Ops/s 718.5770 Ops/s $\textbf{\color{#35bf28}+6.65\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.0502ms 33.0769ms 30.2326 Ops/s 29.6026 Ops/s $\color{#35bf28}+2.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.2734ms 17.6659ms 56.6061 Ops/s 55.6456 Ops/s $\color{#35bf28}+1.73\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.2120ms 33.7401ms 29.6383 Ops/s 28.5471 Ops/s $\color{#35bf28}+3.82\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.1774ms 17.9896ms 55.5878 Ops/s 54.6151 Ops/s $\color{#35bf28}+1.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 37.1420ms 35.5173ms 28.1553 Ops/s 27.5397 Ops/s $\color{#35bf28}+2.24\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.1542ms 19.0114ms 52.6000 Ops/s 49.1857 Ops/s $\textbf{\color{#35bf28}+6.94\%}$

vmoens added 7 commits October 19, 2025 20:56
@vmoens
Update
1591d0e 
[ghstack-poisoned]
@vmoens
Update
2c479eb 
[ghstack-poisoned]
@vmoens
Update
1d4c308 
[ghstack-poisoned]
@vmoens
Update
3092841 
[ghstack-poisoned]
@vmoens
Update
323ed9d 
[ghstack-poisoned]
@vmoens
Update
1cb3227 
[ghstack-poisoned]
@vmoens
Update
826ff12 
[ghstack-poisoned]
This was referenced Oct 22, 2025
Open
Open
vmoens added 7 commits October 22, 2025 12:31
@vmoens
Update
f310600 
[ghstack-poisoned]
@vmoens
Update
f7046b6 
[ghstack-poisoned]
@vmoens
Update
1ff164a 
[ghstack-poisoned]
@vmoens
Update
f7476e7 
[ghstack-poisoned]
@vmoens
Update
ba34fac 
[ghstack-poisoned]
@vmoens
Update
d5dff71 
[ghstack-poisoned]
@vmoens
Update
ffd35d7 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens