🐞 bug report

Affected Rule

pip.parse

Is this a regression?

No

Description

Through various issues it has been recommended that I use experimental_index_url to get improved behavior for pip.parse. My discovery after attempting to use this in our project is that fetching all deps for the first time slowed down significantly. Here are some of the stats I pulled in our project:

without experimental_index_url:

bazel query 'deps(...)' > /dev/null  1.02s user 0.42s system 0% cpu 4:42.53 total
bazel query 'deps(...)' > /dev/null  1.05s user 0.42s system 0% cpu 5:31.24 total

with experimental index

bazel query 'deps(...)' > /dev/null  1.28s user 0.45s system 0% cpu 6:56.47 total
bazel query 'deps(...)' > /dev/null  1.16s user 0.51s system 0% cpu 7:15.13 total
bazel query 'deps(...)' > /dev/null  1.22s user 0.54s system 0% cpu 7:52.11 total

It seems like this is consistent, I tested about 10 times before assuming this was the cause. I imagine this is heavily dependent on what other repo activity exists in the project. I tried messing with the value of --http_max_parallel_downloads without any luck.

To test this I did a rm -rf ~/.cache/pip ~/.cache/bazel between runs to make sure I was starting completely clean.

🔬 Minimal Reproduction

I tried reproducing this in a rules_python example but the difference was not as severe as in our project, which is what makes me think that it's heavily dependent on other http_archives etc that you have.

🌍 Your Environment

Operating System:

  
linux x86_64
  

Output of bazel version:

  
https://github.com/bazelbuild/bazel/commit/2780393d35ad0607cf5e344ae082b00a5569a964
  

Rules_python version:

  
1.4.0-rc
  

Anything else relevant?

https://bazelbuild.slack.com/archives/CA306CEV6/p1745365100854859