Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[BugFix] Test and fix life cycle of env with dynamic non-tensor spec #2812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Feb 26, 2025
Merged

[BugFix] Test and fix life cycle of env with dynamic non-tensor spec #2812

merged 6 commits into from
Feb 26, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Feb 25, 2025 •
edited
Loading

Update
22f7687 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 25, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2812

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit 40a67d7 with merge base d4f8846 (image):

NEW FAILURE - The following job has failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens pushed a commit that referenced this pull request Feb 25, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
de55434 
ghstack-source-id: 9dfc03d
Pull Request resolved: #2812
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 25, 2025
Update
77d7edd 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
f186e21 
ghstack-source-id: 52c8624
Pull Request resolved: #2812
Update
5abb821 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
27784f8 
ghstack-source-id: 524af22
Pull Request resolved: #2812
Update
c9dbda8 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
5fc8670 
ghstack-source-id: 3927e9c
Pull Request resolved: #2812
@github-actions
Copy link

github-actions bot commented Feb 26, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.6652s 0.5325s 1.8780 Ops/s 1.9602 Ops/s $\color{#d91a1a}-4.19\%$
test_transformed 1.0810s 1.0060s 0.9940 Ops/s 0.9622 Ops/s $\color{#35bf28}+3.31\%$
test_serial 1.6464s 1.5295s 0.6538 Ops/s 0.6510 Ops/s $\color{#35bf28}+0.43\%$
test_parallel 1.4331s 1.3373s 0.7478 Ops/s 0.7585 Ops/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-True-True-True-True] 0.3987ms 30.0013μs 33.3319 KOps/s 33.5118 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-True-True-True-False] 56.4140μs 18.0224μs 55.4864 KOps/s 56.1392 KOps/s $\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-True-True-False-True] 45.1750μs 17.0811μs 58.5444 KOps/s 58.3791 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-True-True-False-False] 34.5540μs 10.1343μs 98.6749 KOps/s 98.9510 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-False-True-True] 65.2410μs 31.9501μs 31.2989 KOps/s 31.1797 KOps/s $\color{#35bf28}+0.38\%$
test_step_mdp_speed[True-True-False-True-False] 65.1310μs 19.7908μs 50.5286 KOps/s 50.5470 KOps/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-False-False-True] 45.8760μs 18.9521μs 52.7645 KOps/s 52.1108 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-False-False-False] 49.8330μs 11.9028μs 84.0138 KOps/s 84.0161 KOps/s $-0.00\%$
test_step_mdp_speed[True-False-True-True-True] 66.8940μs 34.0365μs 29.3802 KOps/s 29.3717 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[True-False-True-True-False] 76.6550μs 21.6394μs 46.2119 KOps/s 46.3270 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-False-True-False-True] 0.2077ms 19.0186μs 52.5800 KOps/s 51.9599 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[True-False-True-False-False] 51.2950μs 11.9640μs 83.5838 KOps/s 83.7958 KOps/s $\color{#d91a1a}-0.25\%$
test_step_mdp_speed[True-False-False-True-True] 0.5595ms 35.5702μs 28.1134 KOps/s 27.8882 KOps/s $\color{#35bf28}+0.81\%$
test_step_mdp_speed[True-False-False-True-False] 66.0630μs 23.2102μs 43.0846 KOps/s 42.5422 KOps/s $\color{#35bf28}+1.27\%$
test_step_mdp_speed[True-False-False-False-True] 59.1100μs 20.3817μs 49.0635 KOps/s 47.9051 KOps/s $\color{#35bf28}+2.42\%$
test_step_mdp_speed[True-False-False-False-False] 48.5100μs 13.5802μs 73.6367 KOps/s 71.3966 KOps/s $\color{#35bf28}+3.14\%$
test_step_mdp_speed[False-True-True-True-True] 74.6580μs 33.7161μs 29.6594 KOps/s 29.3718 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[False-True-True-True-False] 58.8890μs 21.5608μs 46.3804 KOps/s 46.1396 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-True-True-False-True] 56.2840μs 21.5633μs 46.3750 KOps/s 45.9347 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[False-True-True-False-False] 62.7570μs 13.2524μs 75.4578 KOps/s 74.5215 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-False-True-True] 77.1440μs 35.6680μs 28.0363 KOps/s 27.9411 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[False-True-False-True-False] 63.4580μs 23.3350μs 42.8540 KOps/s 42.6396 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-True-False-False-True] 2.5236ms 23.5594μs 42.4459 KOps/s 42.6655 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-True-False-False-False] 51.8260μs 15.0029μs 66.6536 KOps/s 66.7088 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-False-True-True-True] 84.0670μs 37.4165μs 26.7262 KOps/s 26.6857 KOps/s $\color{#35bf28}+0.15\%$
test_step_mdp_speed[False-False-True-True-False] 58.5090μs 25.1143μs 39.8179 KOps/s 39.4567 KOps/s $\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-True-False-True] 62.6970μs 23.2936μs 42.9303 KOps/s 42.7055 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-False-True-False-False] 40.6360μs 15.0550μs 66.4230 KOps/s 65.6302 KOps/s $\color{#35bf28}+1.21\%$
test_step_mdp_speed[False-False-False-True-True] 88.7750μs 38.9173μs 25.6955 KOps/s 25.6027 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-True-False] 64.9110μs 26.6923μs 37.4640 KOps/s 37.1871 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-False-False-False-True] 0.5892ms 24.8668μs 40.2143 KOps/s 40.0708 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-False-False] 53.8310μs 16.7109μs 59.8413 KOps/s 60.1228 KOps/s $\color{#d91a1a}-0.47\%$
test_values[generalized_advantage_estimate-True-True] 10.1451ms 9.8679ms 101.3390 Ops/s 99.8187 Ops/s $\color{#35bf28}+1.52\%$
test_values[vec_generalized_advantage_estimate-True-True] 28.2386ms 26.7841ms 37.3356 Ops/s 41.0283 Ops/s $\textbf{\color{#d91a1a}-9.00\%}$
test_values[td0_return_estimate-False-False] 0.2470ms 0.1817ms 5.5023 KOps/s 5.5406 KOps/s $\color{#d91a1a}-0.69\%$
test_values[td1_return_estimate-False-False] 27.2348ms 24.0987ms 41.4960 Ops/s 40.5742 Ops/s $\color{#35bf28}+2.27\%$
test_values[vec_td1_return_estimate-False-False] 28.8042ms 27.0111ms 37.0218 Ops/s 40.8726 Ops/s $\textbf{\color{#d91a1a}-9.42\%}$
test_values[td_lambda_return_estimate-True-False] 37.8077ms 34.6046ms 28.8979 Ops/s 28.3600 Ops/s $\color{#35bf28}+1.90\%$
test_values[vec_td_lambda_return_estimate-True-False] 29.3103ms 27.0867ms 36.9185 Ops/s 41.1224 Ops/s $\textbf{\color{#d91a1a}-10.22\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.1704ms 8.5369ms 117.1384 Ops/s 114.4566 Ops/s $\color{#35bf28}+2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4761ms 1.8771ms 532.7391 Ops/s 548.3392 Ops/s $\color{#d91a1a}-2.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4527ms 0.3644ms 2.7441 KOps/s 2.6481 KOps/s $\color{#35bf28}+3.62\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 44.6122ms 42.7239ms 23.4061 Ops/s 24.1173 Ops/s $\color{#d91a1a}-2.95\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 4.2166ms 3.4253ms 291.9439 Ops/s 289.6244 Ops/s $\color{#35bf28}+0.80\%$
test_dqn_speed[False-None] 6.0136ms 1.4014ms 713.5774 Ops/s 689.2484 Ops/s $\color{#35bf28}+3.53\%$
test_dqn_speed[False-backward] 1.9650ms 1.8732ms 533.8565 Ops/s 511.8990 Ops/s $\color{#35bf28}+4.29\%$
test_dqn_speed[True-None] 0.5908ms 0.4700ms 2.1278 KOps/s 2.0437 KOps/s $\color{#35bf28}+4.11\%$
test_dqn_speed[True-backward] 0.9758ms 0.9172ms 1.0902 KOps/s 1.0834 KOps/s $\color{#35bf28}+0.63\%$
test_dqn_speed[reduce-overhead-None] 0.7378ms 0.4786ms 2.0896 KOps/s 2.0603 KOps/s $\color{#35bf28}+1.42\%$
test_dqn_speed[reduce-overhead-backward] 1.0384ms 0.9081ms 1.1012 KOps/s 1.0117 KOps/s $\textbf{\color{#35bf28}+8.85\%}$
test_ddpg_speed[False-None] 3.6798ms 2.8745ms 347.8910 Ops/s 339.9506 Ops/s $\color{#35bf28}+2.34\%$
test_ddpg_speed[False-backward] 4.0544ms 3.9668ms 252.0930 Ops/s 244.1474 Ops/s $\color{#35bf28}+3.25\%$
test_ddpg_speed[True-None] 1.7511ms 1.2269ms 815.0622 Ops/s 802.7159 Ops/s $\color{#35bf28}+1.54\%$
test_ddpg_speed[True-backward] 2.1927ms 2.1138ms 473.0899 Ops/s 460.2913 Ops/s $\color{#35bf28}+2.78\%$
test_ddpg_speed[reduce-overhead-None] 1.7160ms 1.2291ms 813.6169 Ops/s 812.8908 Ops/s $\color{#35bf28}+0.09\%$
test_ddpg_speed[reduce-overhead-backward] 2.1855ms 2.1068ms 474.6617 Ops/s 465.3582 Ops/s $\color{#35bf28}+2.00\%$
test_sac_speed[False-None] 9.8510ms 7.9153ms 126.3369 Ops/s 122.0702 Ops/s $\color{#35bf28}+3.50\%$
test_sac_speed[False-backward] 11.5991ms 10.6361ms 94.0196 Ops/s 90.8265 Ops/s $\color{#35bf28}+3.52\%$
test_sac_speed[True-None] 2.7005ms 2.1857ms 457.5275 Ops/s 473.4598 Ops/s $\color{#d91a1a}-3.37\%$
test_sac_speed[True-backward] 3.9577ms 3.8396ms 260.4428 Ops/s 254.3472 Ops/s $\color{#35bf28}+2.40\%$
test_sac_speed[reduce-overhead-None] 2.7222ms 2.1933ms 455.9314 Ops/s 472.7784 Ops/s $\color{#d91a1a}-3.56\%$
test_sac_speed[reduce-overhead-backward] 4.0374ms 3.8885ms 257.1717 Ops/s 258.8355 Ops/s $\color{#d91a1a}-0.64\%$
test_redq_speed[False-None] 17.2611ms 13.0477ms 76.6419 Ops/s 74.7550 Ops/s $\color{#35bf28}+2.52\%$
test_redq_speed[False-backward] 30.1377ms 23.1900ms 43.1220 Ops/s 44.2935 Ops/s $\color{#d91a1a}-2.64\%$
test_redq_speed[True-None] 6.5718ms 5.5379ms 180.5732 Ops/s 195.9756 Ops/s $\textbf{\color{#d91a1a}-7.86\%}$
test_redq_speed[True-backward] 14.7256ms 13.6774ms 73.1132 Ops/s 74.0562 Ops/s $\color{#d91a1a}-1.27\%$
test_redq_speed[reduce-overhead-None] 6.7454ms 5.2616ms 190.0577 Ops/s 178.9554 Ops/s $\textbf{\color{#35bf28}+6.20\%}$
test_redq_speed[reduce-overhead-backward] 13.6948ms 12.8496ms 77.8235 Ops/s 78.0355 Ops/s $\color{#d91a1a}-0.27\%$
test_redq_deprec_speed[False-None] 15.1124ms 12.8884ms 77.5891 Ops/s 75.7679 Ops/s $\color{#35bf28}+2.40\%$
test_redq_deprec_speed[False-backward] 20.2102ms 18.6256ms 53.6896 Ops/s 52.3204 Ops/s $\color{#35bf28}+2.62\%$
test_redq_deprec_speed[True-None] 4.6456ms 4.4841ms 223.0100 Ops/s 250.5113 Ops/s $\textbf{\color{#d91a1a}-10.98\%}$
test_redq_deprec_speed[True-backward] 9.5651ms 9.0168ms 110.9038 Ops/s 116.2498 Ops/s $\color{#d91a1a}-4.60\%$
test_redq_deprec_speed[reduce-overhead-None] 5.3497ms 4.5152ms 221.4732 Ops/s 231.0433 Ops/s $\color{#d91a1a}-4.14\%$
test_redq_deprec_speed[reduce-overhead-backward] 10.7775ms 9.5438ms 104.7798 Ops/s 104.8700 Ops/s $\color{#d91a1a}-0.09\%$
test_td3_speed[False-None] 8.7581ms 8.2581ms 121.0933 Ops/s 117.1989 Ops/s $\color{#35bf28}+3.32\%$
test_td3_speed[False-backward] 11.3151ms 10.7131ms 93.3434 Ops/s 89.4984 Ops/s $\color{#35bf28}+4.30\%$
test_td3_speed[True-None] 2.1360ms 1.9489ms 513.1153 Ops/s 537.5718 Ops/s $\color{#d91a1a}-4.55\%$
test_td3_speed[True-backward] 3.9156ms 3.5920ms 278.3934 Ops/s 259.7197 Ops/s $\textbf{\color{#35bf28}+7.19\%}$
test_td3_speed[reduce-overhead-None] 2.1072ms 1.9302ms 518.0711 Ops/s 541.3039 Ops/s $\color{#d91a1a}-4.29\%$
test_td3_speed[reduce-overhead-backward] 3.6510ms 3.5870ms 278.7828 Ops/s 285.3425 Ops/s $\color{#d91a1a}-2.30\%$
test_cql_speed[False-None] 38.0115ms 36.6027ms 27.3204 Ops/s 26.0760 Ops/s $\color{#35bf28}+4.77\%$
test_cql_speed[False-backward] 50.8872ms 47.8015ms 20.9198 Ops/s 20.7711 Ops/s $\color{#35bf28}+0.72\%$
test_cql_speed[True-None] 17.9078ms 17.0071ms 58.7988 Ops/s 59.5348 Ops/s $\color{#d91a1a}-1.24\%$
test_cql_speed[True-backward] 25.2924ms 23.9168ms 41.8116 Ops/s 43.2350 Ops/s $\color{#d91a1a}-3.29\%$
test_cql_speed[reduce-overhead-None] 18.0722ms 17.2254ms 58.0539 Ops/s 59.3129 Ops/s $\color{#d91a1a}-2.12\%$
test_cql_speed[reduce-overhead-backward] 25.5742ms 24.7648ms 40.3798 Ops/s 43.1244 Ops/s $\textbf{\color{#d91a1a}-6.36\%}$
test_a2c_speed[False-None] 9.2867ms 7.7498ms 129.0351 Ops/s 137.3447 Ops/s $\textbf{\color{#d91a1a}-6.05\%}$
test_a2c_speed[False-backward] 17.0884ms 15.3376ms 65.1990 Ops/s 69.8289 Ops/s $\textbf{\color{#d91a1a}-6.63\%}$
test_a2c_speed[True-None] 4.1504ms 3.8315ms 260.9946 Ops/s 267.0491 Ops/s $\color{#d91a1a}-2.27\%$
test_a2c_speed[True-backward] 11.1080ms 10.6177ms 94.1822 Ops/s 96.4295 Ops/s $\color{#d91a1a}-2.33\%$
test_a2c_speed[reduce-overhead-None] 4.4951ms 3.7466ms 266.9065 Ops/s 265.5201 Ops/s $\color{#35bf28}+0.52\%$
test_a2c_speed[reduce-overhead-backward] 10.4889ms 10.1146ms 98.8669 Ops/s 97.0923 Ops/s $\color{#35bf28}+1.83\%$
test_ppo_speed[False-None] 8.6621ms 7.4730ms 133.8159 Ops/s 129.9490 Ops/s $\color{#35bf28}+2.98\%$
test_ppo_speed[False-backward] 15.2667ms 14.8379ms 67.3951 Ops/s 65.4995 Ops/s $\color{#35bf28}+2.89\%$
test_ppo_speed[True-None] 5.5010ms 4.1262ms 242.3534 Ops/s 238.1598 Ops/s $\color{#35bf28}+1.76\%$
test_ppo_speed[True-backward] 11.0635ms 10.4522ms 95.6739 Ops/s 98.3101 Ops/s $\color{#d91a1a}-2.68\%$
test_ppo_speed[reduce-overhead-None] 4.4501ms 4.1268ms 242.3167 Ops/s 238.3865 Ops/s $\color{#35bf28}+1.65\%$
test_ppo_speed[reduce-overhead-backward] 11.0136ms 10.1339ms 98.6789 Ops/s 98.3516 Ops/s $\color{#35bf28}+0.33\%$
test_reinforce_speed[False-None] 7.3912ms 6.5795ms 151.9863 Ops/s 148.7777 Ops/s $\color{#35bf28}+2.16\%$
test_reinforce_speed[False-backward] 10.2678ms 9.9751ms 100.2497 Ops/s 98.1693 Ops/s $\color{#35bf28}+2.12\%$
test_reinforce_speed[True-None] 3.4670ms 3.1081ms 321.7435 Ops/s 317.2041 Ops/s $\color{#35bf28}+1.43\%$
test_reinforce_speed[True-backward] 9.9242ms 9.2541ms 108.0602 Ops/s 109.7668 Ops/s $\color{#d91a1a}-1.55\%$
test_reinforce_speed[reduce-overhead-None] 3.3961ms 3.0705ms 325.6758 Ops/s 314.9208 Ops/s $\color{#35bf28}+3.42\%$
test_reinforce_speed[reduce-overhead-backward] 12.9617ms 9.1601ms 109.1696 Ops/s 103.5475 Ops/s $\textbf{\color{#35bf28}+5.43\%}$
test_iql_speed[False-None] 39.8885ms 32.3019ms 30.9580 Ops/s 29.7320 Ops/s $\color{#35bf28}+4.12\%$
test_iql_speed[False-backward] 46.9659ms 44.7385ms 22.3521 Ops/s 21.2665 Ops/s $\textbf{\color{#35bf28}+5.10\%}$
test_iql_speed[True-None] 13.0968ms 11.8237ms 84.5758 Ops/s 88.0627 Ops/s $\color{#d91a1a}-3.96\%$
test_iql_speed[True-backward] 23.5872ms 22.3998ms 44.6432 Ops/s 44.8897 Ops/s $\color{#d91a1a}-0.55\%$
test_iql_speed[reduce-overhead-None] 13.3858ms 11.6303ms 85.9826 Ops/s 87.6012 Ops/s $\color{#d91a1a}-1.85\%$
test_iql_speed[reduce-overhead-backward] 23.1484ms 22.4239ms 44.5953 Ops/s 43.7579 Ops/s $\color{#35bf28}+1.91\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3154ms 4.7013ms 212.7083 Ops/s 204.7548 Ops/s $\color{#35bf28}+3.88\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8320ms 0.5334ms 1.8746 KOps/s 1.8370 KOps/s $\color{#35bf28}+2.05\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9198ms 0.5080ms 1.9684 KOps/s 1.9504 KOps/s $\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.1597ms 4.5074ms 221.8580 Ops/s 214.5662 Ops/s $\color{#35bf28}+3.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.5827ms 0.5221ms 1.9154 KOps/s 1.8817 KOps/s $\color{#35bf28}+1.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8694ms 0.4990ms 2.0040 KOps/s 1.9499 KOps/s $\color{#35bf28}+2.77\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.6734ms 1.7085ms 585.2991 Ops/s 572.4174 Ops/s $\color{#35bf28}+2.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3652ms 1.6862ms 593.0568 Ops/s 586.6518 Ops/s $\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.9213ms 4.6189ms 216.4999 Ops/s 200.5427 Ops/s $\textbf{\color{#35bf28}+7.96\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.2508ms 0.6652ms 1.5032 KOps/s 1.4343 KOps/s $\color{#35bf28}+4.80\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0281ms 0.6424ms 1.5568 KOps/s 1.5215 KOps/s $\color{#35bf28}+2.32\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.6498ms 4.4742ms 223.5033 Ops/s 206.9316 Ops/s $\textbf{\color{#35bf28}+8.01\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.4016ms 0.5380ms 1.8587 KOps/s 1.8037 KOps/s $\color{#35bf28}+3.05\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7946ms 0.5069ms 1.9728 KOps/s 1.9208 KOps/s $\color{#35bf28}+2.71\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1293ms 4.4565ms 224.3938 Ops/s 204.6070 Ops/s $\textbf{\color{#35bf28}+9.67\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.3367ms 0.5247ms 1.9058 KOps/s 1.8078 KOps/s $\textbf{\color{#35bf28}+5.42\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9158ms 0.5031ms 1.9876 KOps/s 1.9588 KOps/s $\color{#35bf28}+1.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.2767ms 4.6704ms 214.1142 Ops/s 202.1697 Ops/s $\textbf{\color{#35bf28}+5.91\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.5297ms 0.6739ms 1.4839 KOps/s 1.4514 KOps/s $\color{#35bf28}+2.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8316ms 0.6369ms 1.5700 KOps/s 1.5152 KOps/s $\color{#35bf28}+3.62\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 5.5424ms 4.2226ms 236.8193 Ops/s 225.6497 Ops/s $\color{#35bf28}+4.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 5.8871ms 2.3868ms 418.9690 Ops/s 414.0910 Ops/s $\color{#35bf28}+1.18\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2290ms 1.4791ms 676.1078 Ops/s 755.7032 Ops/s $\textbf{\color{#d91a1a}-10.53\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4455s 13.1462ms 76.0676 Ops/s 222.3208 Ops/s $\textbf{\color{#d91a1a}-65.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 7.8274ms 2.3993ms 416.7918 Ops/s 415.1587 Ops/s $\color{#35bf28}+0.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.7399ms 1.4216ms 703.4514 Ops/s 744.9040 Ops/s $\textbf{\color{#d91a1a}-5.56\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 5.8057ms 4.4950ms 222.4678 Ops/s 227.2468 Ops/s $\color{#d91a1a}-2.10\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.3004ms 2.5385ms 393.9266 Ops/s 384.7110 Ops/s $\color{#35bf28}+2.40\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.8014ms 1.5002ms 666.5909 Ops/s 586.2158 Ops/s $\textbf{\color{#35bf28}+13.71\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 12.3952ms 11.7606ms 85.0296 Ops/s 77.5775 Ops/s $\textbf{\color{#35bf28}+9.61\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 15.5988ms 14.4414ms 69.2452 Ops/s 68.4883 Ops/s $\color{#35bf28}+1.11\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 21.8186ms 20.5345ms 48.6986 Ops/s 46.9526 Ops/s $\color{#35bf28}+3.72\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.6940ms 14.5093ms 68.9214 Ops/s 65.5779 Ops/s $\textbf{\color{#35bf28}+5.10\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 21.9742ms 20.5582ms 48.6424 Ops/s 46.3577 Ops/s $\color{#35bf28}+4.93\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 17.0545ms 15.9244ms 62.7969 Ops/s 61.2048 Ops/s $\color{#35bf28}+2.60\%$

@github-actions
Copy link

github-actions bot commented Feb 26, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.8928s 0.8094s 1.2354 Ops/s 1.2138 Ops/s $\color{#35bf28}+1.79\%$
test_transformed 1.3756s 1.3737s 0.7280 Ops/s 0.6931 Ops/s $\textbf{\color{#35bf28}+5.04\%}$
test_serial 2.2555s 2.2480s 0.4448 Ops/s 0.4278 Ops/s $\color{#35bf28}+3.99\%$
test_parallel 1.8989s 1.8398s 0.5435 Ops/s 0.5330 Ops/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-True-True-True-True] 0.2440ms 38.8201μs 25.7598 KOps/s 25.9706 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-True-True-True-False] 53.5210μs 22.4190μs 44.6050 KOps/s 45.0732 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-True-True-False-True] 56.2410μs 21.4286μs 46.6666 KOps/s 49.0785 KOps/s $\color{#d91a1a}-4.91\%$
test_step_mdp_speed[True-True-True-False-False] 46.5810μs 12.5105μs 79.9328 KOps/s 81.8337 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[True-True-False-True-True] 77.2620μs 40.8498μs 24.4799 KOps/s 24.7044 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-True-False-True-False] 60.6910μs 24.9215μs 40.1259 KOps/s 41.0723 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-True-False-False-True] 52.4810μs 23.6657μs 42.2553 KOps/s 42.6199 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[True-True-False-False-False] 45.7010μs 15.0296μs 66.5353 KOps/s 68.8153 KOps/s $\color{#d91a1a}-3.31\%$
test_step_mdp_speed[True-False-True-True-True] 86.5120μs 43.4151μs 23.0335 KOps/s 23.2110 KOps/s $\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-False-True-True-False] 57.4610μs 27.0008μs 37.0359 KOps/s 37.8504 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[True-False-True-False-True] 60.2210μs 24.0227μs 41.6274 KOps/s 43.0390 KOps/s $\color{#d91a1a}-3.28\%$
test_step_mdp_speed[True-False-True-False-False] 41.9400μs 14.8167μs 67.4912 KOps/s 69.0927 KOps/s $\color{#d91a1a}-2.32\%$
test_step_mdp_speed[True-False-False-True-True] 81.8020μs 45.7899μs 21.8389 KOps/s 22.3836 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-False-False-True-False] 67.3710μs 28.9022μs 34.5994 KOps/s 34.6337 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[True-False-False-False-True] 59.3710μs 25.9694μs 38.5069 KOps/s 38.8178 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-False-False-False-False] 47.7410μs 17.1861μs 58.1866 KOps/s 59.7528 KOps/s $\color{#d91a1a}-2.62\%$
test_step_mdp_speed[False-True-True-True-True] 75.5320μs 43.1112μs 23.1958 KOps/s 23.3617 KOps/s $\color{#d91a1a}-0.71\%$
test_step_mdp_speed[False-True-True-True-False] 69.1910μs 27.2389μs 36.7122 KOps/s 37.4138 KOps/s $\color{#d91a1a}-1.88\%$
test_step_mdp_speed[False-True-True-False-True] 2.7256ms 28.2212μs 35.4344 KOps/s 36.1825 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[False-True-True-False-False] 67.8610μs 16.7399μs 59.7375 KOps/s 61.6895 KOps/s $\color{#d91a1a}-3.16\%$
test_step_mdp_speed[False-True-False-True-True] 74.4420μs 46.0337μs 21.7232 KOps/s 22.4089 KOps/s $\color{#d91a1a}-3.06\%$
test_step_mdp_speed[False-True-False-True-False] 67.4510μs 29.5094μs 33.8875 KOps/s 35.3212 KOps/s $\color{#d91a1a}-4.06\%$
test_step_mdp_speed[False-True-False-False-True] 71.3210μs 29.7008μs 33.6691 KOps/s 34.6288 KOps/s $\color{#d91a1a}-2.77\%$
test_step_mdp_speed[False-True-False-False-False] 65.8220μs 18.8516μs 53.0460 KOps/s 53.7768 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-False-True-True-True] 87.3910μs 48.2186μs 20.7389 KOps/s 21.2810 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-False-True-True-False] 68.4210μs 31.4375μs 31.8091 KOps/s 32.2735 KOps/s $\color{#d91a1a}-1.44\%$
test_step_mdp_speed[False-False-True-False-True] 57.8110μs 29.8047μs 33.5518 KOps/s 34.5699 KOps/s $\color{#d91a1a}-2.94\%$
test_step_mdp_speed[False-False-True-False-False] 51.6310μs 18.9095μs 52.8834 KOps/s 54.2653 KOps/s $\color{#d91a1a}-2.55\%$
test_step_mdp_speed[False-False-False-True-True] 80.1810μs 49.5499μs 20.1817 KOps/s 20.6194 KOps/s $\color{#d91a1a}-2.12\%$
test_step_mdp_speed[False-False-False-True-False] 60.4910μs 34.0603μs 29.3597 KOps/s 29.9783 KOps/s $\color{#d91a1a}-2.06\%$
test_step_mdp_speed[False-False-False-False-True] 63.1310μs 31.7213μs 31.5245 KOps/s 32.7561 KOps/s $\color{#d91a1a}-3.76\%$
test_step_mdp_speed[False-False-False-False-False] 50.5210μs 21.1751μs 47.2252 KOps/s 48.3459 KOps/s $\color{#d91a1a}-2.32\%$
test_values[generalized_advantage_estimate-True-True] 25.0007ms 24.5357ms 40.7569 Ops/s 39.4718 Ops/s $\color{#35bf28}+3.26\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1190s 3.2782ms 305.0427 Ops/s 337.5657 Ops/s $\textbf{\color{#d91a1a}-9.63\%}$
test_values[td0_return_estimate-False-False] 0.1054ms 79.9544μs 12.5071 KOps/s 12.3822 KOps/s $\color{#35bf28}+1.01\%$
test_values[td1_return_estimate-False-False] 56.1176ms 54.9135ms 18.2105 Ops/s 17.5090 Ops/s $\color{#35bf28}+4.01\%$
test_values[vec_td1_return_estimate-False-False] 1.3135ms 1.0831ms 923.2627 Ops/s 911.5939 Ops/s $\color{#35bf28}+1.28\%$
test_values[td_lambda_return_estimate-True-False] 87.5935ms 87.2131ms 11.4662 Ops/s 11.0276 Ops/s $\color{#35bf28}+3.98\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3947ms 1.0863ms 920.5217 Ops/s 925.5964 Ops/s $\color{#d91a1a}-0.55\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.5848ms 24.4211ms 40.9482 Ops/s 38.8887 Ops/s $\textbf{\color{#35bf28}+5.30\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0332ms 0.7513ms 1.3310 KOps/s 1.3265 KOps/s $\color{#35bf28}+0.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7547ms 0.6679ms 1.4973 KOps/s 1.4600 KOps/s $\color{#35bf28}+2.56\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5173ms 1.4792ms 676.0398 Ops/s 673.8288 Ops/s $\color{#35bf28}+0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8550ms 0.6901ms 1.4492 KOps/s 1.4609 KOps/s $\color{#d91a1a}-0.80\%$
test_dqn_speed[False-None] 6.9843ms 1.4754ms 677.7844 Ops/s 670.3001 Ops/s $\color{#35bf28}+1.12\%$
test_dqn_speed[False-backward] 2.2542ms 2.0819ms 480.3379 Ops/s 473.0537 Ops/s $\color{#35bf28}+1.54\%$
test_dqn_speed[True-None] 0.5900ms 0.5363ms 1.8645 KOps/s 1.7934 KOps/s $\color{#35bf28}+3.97\%$
test_dqn_speed[True-backward] 1.2481ms 1.1979ms 834.7865 Ops/s 890.4806 Ops/s $\textbf{\color{#d91a1a}-6.25\%}$
test_dqn_speed[reduce-overhead-None] 0.6901ms 0.5610ms 1.7827 KOps/s 1.7341 KOps/s $\color{#35bf28}+2.80\%$
test_dqn_speed[reduce-overhead-backward] 1.1008ms 1.0490ms 953.2492 Ops/s 1.0486 KOps/s $\textbf{\color{#d91a1a}-9.10\%}$
test_ddpg_speed[False-None] 3.0671ms 2.7773ms 360.0601 Ops/s 355.0406 Ops/s $\color{#35bf28}+1.41\%$
test_ddpg_speed[False-backward] 4.5602ms 4.1541ms 240.7253 Ops/s 246.3191 Ops/s $\color{#d91a1a}-2.27\%$
test_ddpg_speed[True-None] 1.5487ms 1.3052ms 766.1406 Ops/s 762.3701 Ops/s $\color{#35bf28}+0.49\%$
test_ddpg_speed[True-backward] 2.5473ms 2.5158ms 397.4835 Ops/s 415.5390 Ops/s $\color{#d91a1a}-4.35\%$
test_ddpg_speed[reduce-overhead-None] 1.3535ms 1.3074ms 764.8644 Ops/s 753.3812 Ops/s $\color{#35bf28}+1.52\%$
test_ddpg_speed[reduce-overhead-backward] 2.0416ms 1.9890ms 502.7736 Ops/s 533.7906 Ops/s $\textbf{\color{#d91a1a}-5.81\%}$
test_sac_speed[False-None] 8.3284ms 7.9377ms 125.9812 Ops/s 123.9362 Ops/s $\color{#35bf28}+1.65\%$
test_sac_speed[False-backward] 11.8891ms 11.1219ms 89.9127 Ops/s 90.9767 Ops/s $\color{#d91a1a}-1.17\%$
test_sac_speed[True-None] 2.3452ms 1.8019ms 554.9552 Ops/s 551.2687 Ops/s $\color{#35bf28}+0.67\%$
test_sac_speed[True-backward] 3.7448ms 3.6874ms 271.1917 Ops/s 267.8385 Ops/s $\color{#35bf28}+1.25\%$
test_sac_speed[reduce-overhead-None] 20.7377ms 11.7734ms 84.9370 Ops/s 84.7679 Ops/s $\color{#35bf28}+0.20\%$
test_sac_speed[reduce-overhead-backward] 1.7911ms 1.7429ms 573.7500 Ops/s 559.4900 Ops/s $\color{#35bf28}+2.55\%$
test_redq_speed[False-None] 7.9488ms 7.4716ms 133.8405 Ops/s 130.9029 Ops/s $\color{#35bf28}+2.24\%$
test_redq_speed[False-backward] 12.1529ms 11.6746ms 85.6561 Ops/s 84.6924 Ops/s $\color{#35bf28}+1.14\%$
test_redq_speed[True-None] 2.3619ms 2.2880ms 437.0616 Ops/s 428.9388 Ops/s $\color{#35bf28}+1.89\%$
test_redq_speed[True-backward] 4.1883ms 4.1477ms 241.0984 Ops/s 238.2394 Ops/s $\color{#35bf28}+1.20\%$
test_redq_speed[reduce-overhead-None] 2.4480ms 2.3081ms 433.2491 Ops/s 428.7658 Ops/s $\color{#35bf28}+1.05\%$
test_redq_speed[reduce-overhead-backward] 4.2503ms 4.1521ms 240.8440 Ops/s 235.7758 Ops/s $\color{#35bf28}+2.15\%$
test_redq_deprec_speed[False-None] 9.4989ms 8.9835ms 111.3156 Ops/s 109.8103 Ops/s $\color{#35bf28}+1.37\%$
test_redq_deprec_speed[False-backward] 12.7423ms 12.2376ms 81.7151 Ops/s 80.3694 Ops/s $\color{#35bf28}+1.67\%$
test_redq_deprec_speed[True-None] 2.7565ms 2.5943ms 385.4640 Ops/s 380.7062 Ops/s $\color{#35bf28}+1.25\%$
test_redq_deprec_speed[True-backward] 4.8204ms 4.4185ms 226.3189 Ops/s 230.8123 Ops/s $\color{#d91a1a}-1.95\%$
test_redq_deprec_speed[reduce-overhead-None] 2.6892ms 2.5906ms 386.0113 Ops/s 377.5987 Ops/s $\color{#35bf28}+2.23\%$
test_redq_deprec_speed[reduce-overhead-backward] 4.8738ms 4.4172ms 226.3876 Ops/s 231.5383 Ops/s $\color{#d91a1a}-2.22\%$
test_td3_speed[False-None] 7.8783ms 7.8200ms 127.8770 Ops/s 124.7101 Ops/s $\color{#35bf28}+2.54\%$
test_td3_speed[False-backward] 11.1554ms 10.4142ms 96.0225 Ops/s 98.0748 Ops/s $\color{#d91a1a}-2.09\%$
test_td3_speed[True-None] 1.6173ms 1.5880ms 629.7346 Ops/s 618.8679 Ops/s $\color{#35bf28}+1.76\%$
test_td3_speed[True-backward] 3.3239ms 3.2573ms 306.9992 Ops/s 315.6646 Ops/s $\color{#d91a1a}-2.75\%$
test_td3_speed[reduce-overhead-None] 49.1381ms 24.9943ms 40.0091 Ops/s 41.0229 Ops/s $\color{#d91a1a}-2.47\%$
test_td3_speed[reduce-overhead-backward] 1.5117ms 1.4483ms 690.4837 Ops/s 764.1256 Ops/s $\textbf{\color{#d91a1a}-9.64\%}$
test_cql_speed[False-None] 17.3136ms 16.5294ms 60.4983 Ops/s 59.9713 Ops/s $\color{#35bf28}+0.88\%$
test_cql_speed[False-backward] 22.4164ms 21.9824ms 45.4910 Ops/s 45.6610 Ops/s $\color{#d91a1a}-0.37\%$
test_cql_speed[True-None] 3.6006ms 3.1827ms 314.1965 Ops/s 309.8650 Ops/s $\color{#35bf28}+1.40\%$
test_cql_speed[True-backward] 6.1787ms 5.6402ms 177.2973 Ops/s 183.4576 Ops/s $\color{#d91a1a}-3.36\%$
test_cql_speed[reduce-overhead-None] 21.0287ms 12.7909ms 78.1803 Ops/s 77.0909 Ops/s $\color{#35bf28}+1.41\%$
test_cql_speed[reduce-overhead-backward] 1.9295ms 1.7876ms 559.4177 Ops/s 548.3151 Ops/s $\color{#35bf28}+2.02\%$
test_a2c_speed[False-None] 3.2000ms 3.1106ms 321.4799 Ops/s 303.6143 Ops/s $\textbf{\color{#35bf28}+5.88\%}$
test_a2c_speed[False-backward] 6.5819ms 5.9981ms 166.7198 Ops/s 163.0569 Ops/s $\color{#35bf28}+2.25\%$
test_a2c_speed[True-None] 1.4558ms 1.3134ms 761.3748 Ops/s 749.6976 Ops/s $\color{#35bf28}+1.56\%$
test_a2c_speed[True-backward] 3.0120ms 2.8628ms 349.3135 Ops/s 345.6549 Ops/s $\color{#35bf28}+1.06\%$
test_a2c_speed[reduce-overhead-None] 15.4666ms 8.7678ms 114.0534 Ops/s 117.7702 Ops/s $\color{#d91a1a}-3.16\%$
test_a2c_speed[reduce-overhead-backward] 1.5111ms 1.4349ms 696.9348 Ops/s 692.0390 Ops/s $\color{#35bf28}+0.71\%$
test_ppo_speed[False-None] 3.6995ms 3.6101ms 276.9992 Ops/s 263.5943 Ops/s $\textbf{\color{#35bf28}+5.09\%}$
test_ppo_speed[False-backward] 7.1774ms 6.7092ms 149.0486 Ops/s 145.0626 Ops/s $\color{#35bf28}+2.75\%$
test_ppo_speed[True-None] 1.5058ms 1.3826ms 723.2519 Ops/s 710.1370 Ops/s $\color{#35bf28}+1.85\%$
test_ppo_speed[True-backward] 3.0549ms 3.0100ms 332.2291 Ops/s 324.5571 Ops/s $\color{#35bf28}+2.36\%$
test_ppo_speed[reduce-overhead-None] 1.0730ms 0.9424ms 1.0611 KOps/s 1.0581 KOps/s $\color{#35bf28}+0.29\%$
test_ppo_speed[reduce-overhead-backward] 1.4738ms 1.3782ms 725.5802 Ops/s 691.9241 Ops/s $\color{#35bf28}+4.86\%$
test_reinforce_speed[False-None] 2.3889ms 2.2124ms 452.0069 Ops/s 439.8093 Ops/s $\color{#35bf28}+2.77\%$
test_reinforce_speed[False-backward] 3.2516ms 3.1963ms 312.8656 Ops/s 305.6048 Ops/s $\color{#35bf28}+2.38\%$
test_reinforce_speed[True-None] 1.4056ms 1.2612ms 792.9028 Ops/s 775.0579 Ops/s $\color{#35bf28}+2.30\%$
test_reinforce_speed[True-backward] 3.0002ms 2.8797ms 347.2549 Ops/s 331.1132 Ops/s $\color{#35bf28}+4.87\%$
test_reinforce_speed[reduce-overhead-None] 18.1571ms 9.8776ms 101.2395 Ops/s 102.3669 Ops/s $\color{#d91a1a}-1.10\%$
test_reinforce_speed[reduce-overhead-backward] 1.5820ms 1.4772ms 676.9774 Ops/s 603.5462 Ops/s $\textbf{\color{#35bf28}+12.17\%}$
test_iql_speed[False-None] 9.5404ms 9.0738ms 110.2073 Ops/s 108.0873 Ops/s $\color{#35bf28}+1.96\%$
test_iql_speed[False-backward] 13.2518ms 12.6897ms 78.8039 Ops/s 75.7187 Ops/s $\color{#35bf28}+4.07\%$
test_iql_speed[True-None] 2.4121ms 2.1816ms 458.3833 Ops/s 446.9488 Ops/s $\color{#35bf28}+2.56\%$
test_iql_speed[True-backward] 4.8317ms 4.6429ms 215.3813 Ops/s 202.4064 Ops/s $\textbf{\color{#35bf28}+6.41\%}$
test_iql_speed[reduce-overhead-None] 0.4794s 12.5684ms 79.5645 Ops/s 92.9472 Ops/s $\textbf{\color{#d91a1a}-14.40\%}$
test_iql_speed[reduce-overhead-backward] 1.9413ms 1.8667ms 535.7179 Ops/s 480.4800 Ops/s $\textbf{\color{#35bf28}+11.50\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4687ms 6.0281ms 165.8890 Ops/s 162.2281 Ops/s $\color{#35bf28}+2.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6221ms 0.3351ms 2.9838 KOps/s 3.5096 KOps/s $\textbf{\color{#d91a1a}-14.98\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6181ms 0.3250ms 3.0773 KOps/s 3.9756 KOps/s $\textbf{\color{#d91a1a}-22.60\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9313ms 5.7157ms 174.9568 Ops/s 171.6602 Ops/s $\color{#35bf28}+1.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8088ms 0.2696ms 3.7089 KOps/s 3.4651 KOps/s $\textbf{\color{#35bf28}+7.03\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6209ms 0.3241ms 3.0853 KOps/s 3.2124 KOps/s $\color{#d91a1a}-3.96\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6253ms 1.4168ms 705.8366 Ops/s 787.3403 Ops/s $\textbf{\color{#d91a1a}-10.35\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6975ms 1.2876ms 776.6418 Ops/s 778.1803 Ops/s $\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0854ms 5.9138ms 169.0949 Ops/s 165.8877 Ops/s $\color{#35bf28}+1.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0412ms 0.4744ms 2.1079 KOps/s 2.2143 KOps/s $\color{#d91a1a}-4.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7015ms 0.4459ms 2.2426 KOps/s 2.3717 KOps/s $\textbf{\color{#d91a1a}-5.44\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1278ms 5.7337ms 174.4074 Ops/s 169.0935 Ops/s $\color{#35bf28}+3.14\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7941ms 0.3161ms 3.1636 KOps/s 2.8408 KOps/s $\textbf{\color{#35bf28}+11.36\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5539ms 0.2901ms 3.4466 KOps/s 3.0408 KOps/s $\textbf{\color{#35bf28}+13.34\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.4545ms 5.6719ms 176.3091 Ops/s 171.3811 Ops/s $\color{#35bf28}+2.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8133ms 0.3142ms 3.1824 KOps/s 3.3374 KOps/s $\color{#d91a1a}-4.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6543ms 0.2668ms 3.7476 KOps/s 3.7607 KOps/s $\color{#d91a1a}-0.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0693ms 5.9302ms 168.6297 Ops/s 166.2829 Ops/s $\color{#35bf28}+1.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9405ms 0.4708ms 2.1242 KOps/s 2.3948 KOps/s $\textbf{\color{#d91a1a}-11.30\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7135ms 0.4630ms 2.1599 KOps/s 2.5441 KOps/s $\textbf{\color{#d91a1a}-15.10\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.0948ms 5.3836ms 185.7510 Ops/s 180.2087 Ops/s $\color{#35bf28}+3.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 6.3327ms 2.0119ms 497.0360 Ops/s 442.4566 Ops/s $\textbf{\color{#35bf28}+12.34\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.9246ms 1.2780ms 782.4476 Ops/s 772.9298 Ops/s $\color{#35bf28}+1.23\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4533s 14.4719ms 69.0996 Ops/s 181.2244 Ops/s $\textbf{\color{#d91a1a}-61.87\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.3273ms 1.9750ms 506.3232 Ops/s 435.9056 Ops/s $\textbf{\color{#35bf28}+16.15\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.8194ms 1.2448ms 803.3388 Ops/s 832.1780 Ops/s $\color{#d91a1a}-3.47\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.5374ms 5.6412ms 177.2681 Ops/s 31.3608 Ops/s $\textbf{\color{#35bf28}+465.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.7470ms 2.1821ms 458.2736 Ops/s 447.4130 Ops/s $\color{#35bf28}+2.43\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.2657ms 1.3939ms 717.4184 Ops/s 745.5962 Ops/s $\color{#d91a1a}-3.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.8728ms 13.1607ms 75.9839 Ops/s 73.0186 Ops/s $\color{#35bf28}+4.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.0303ms 16.4028ms 60.9651 Ops/s 60.4841 Ops/s $\color{#35bf28}+0.80\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 18.0292ms 17.7308ms 56.3992 Ops/s 55.0614 Ops/s $\color{#35bf28}+2.43\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 18.8579ms 16.6835ms 59.9395 Ops/s 59.9650 Ops/s $\color{#d91a1a}-0.04\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 0.3863s 25.1411ms 39.7756 Ops/s 55.4756 Ops/s $\textbf{\color{#d91a1a}-28.30\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 19.9864ms 18.6204ms 53.7045 Ops/s 55.5536 Ops/s $\color{#d91a1a}-3.33\%$

@vmoens vmoens added the bug Something isn't working label Feb 26, 2025
Update
33fedaa 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
9b455b1 
ghstack-source-id: cf3fb60
Pull Request resolved: #2812
Update
40a67d7 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
b89ef84 
ghstack-source-id: 77da3a6
Pull Request resolved: #2812
@vmoens vmoens merged commit 40a67d7 into gh/vmoens/92/base Feb 26, 2025
73 of 76 checks passed
vmoens pushed a commit that referenced this pull request Feb 26, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
b538c66 
ghstack-source-id: 77da3a6
Pull Request resolved: #2812
@vmoens vmoens deleted the gh/vmoens/92/head branch February 26, 2025 15:37
vmoens pushed a commit that referenced this pull request Mar 8, 2025
[BugFix] Test and fix life cycle of env with dynamic non-tensor spec
ca5878f 
ghstack-source-id: 77da3a6
Pull Request resolved: #2812

(cherry picked from commit b538c66)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens @facebook-github-bot