I think that there is some value to the additional complexity for controlling the supply chain more aggressively.

I see, would you be more favorable to have a try/catch strategy, sort of like 345452c?

We can:

1. try to install the wheel.
2. if it fails, fallback to a regular pip install.
3. block the regular pip install if an env variable is set in the case we want extra control over the supply chain.

Another option would be to vendor our own wheel for arm64-windows.

What do you think?

Would it be possible to just add a wheel and specify the locations for ARM64 like we do other tools?

Unfortunately psutil does not build a wheel for arm64 windows: https://pypi.org/project/psutil/#files.

Ugh, that is unfortunate. There are separate projects (e.g. https://github.com/cgohlke/win_arm64-wheels) to create wheels for Windows ARM64. The psutil wheel particularly is only needed for testing LLDB, which would be good to do.

I'd love to get @shahmishal's take on this.

@compnerd @shahmishal Would using pip install but constraining it to a known version be a reasonable compromise to provide some control about what we install?

I think that we should absolutely constrain the version. I also wondering if we should be doing more to ensure that something else isn't being injected. If you note the existing implementation, we have a specific URL and expected SHA256 that we compare against.

A quick web search seems to indicate that this might be possible with pip: https://pip.pypa.io/en/stable/topics/secure-installs/

pip install with versions (and hashes if you wish) sounds great! Thanks for working on this.

Oh, nice, I didn't know about that feature! Versions, Hash Checking Mode, and binary only combined would be great.

This approach combines the best of both worlds:

• If a wheel exists, use it and check its hash.
• Otherwise, install from source and check the hash through pip using --requires-hashes.

We have to use a temporary requirements.txt file to pass the hash as the cli does not support it already. There is a tracking issue but it appears to be stale: pypa/pip#3257.

I'm not sure about the source distribution - this will add additional CI time and we are working to reduce that. If you want to use the from source distributions, it will require that we find some way to reduce the build times first.

The source distributions are only used as a fallback. All the modules should be installed from the wheel (no build time), except for psutil on Windows arm64, because there are no wheels for it.

The only way I see to not use a source distribution for psutil on Windows arm64 would be to build a wheel ourselves and host is somewhere or vendor it inside of the repository.

I think there are 3 approaches to fix this script failure on Windows ARM64:

1. Build psutil from source

We use pip install instead of downloading the wheel manually. We use --requires-hashes to make sure the source has not been tempered with.

Pros: Simple to implement, no need to host a file for win-arm64.
Cons: Could slow down the CI, although it takes only 4s to install on my machine (MacBook Pro M2 running Parallels), see below:

(venv) PS C:\Users\charleszablit> Measure-Command { pip install psutil==6.1.0 --no-cache-dir }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 3
Milliseconds      : 740
Ticks             : 37405082
TotalDays         : 4.32929189814815E-05
TotalHours        : 0.00103903005555556
TotalMinutes      : 0.0623418033333333
TotalSeconds      : 3.7405082
TotalMilliseconds : 3740.5082

2. Use a pre-built wheel online

We use the v2024.11.3 release of https://github.com/cgohlke/win_arm64-wheels/, which has a win-arm64 wheel for psutil==6.1.0.

Pros: We don't have to build/host a wheel.
Cons: The release is a zip of all the wheels and has a size of ~350MB, which has to be downloaded then extracted to then be able to install the wheel. This might be slower than building from source.

3. Vendor our own wheel

I was able to build a wheel for psutil==6.1.0 on Windows ARM64. We can host it somewhere or include it in the repo.

Pros: The wheel is available without the need to download.
Cons: We either have to host the file somewhere or include it in the repo (~250KB).

Conclusion

All 3 approaches ensure that we maintain control of the supply chain through checksum checking.
Approach 2 seems to be the slowest due to the 350MB download.
I have a slight preference for 1 over 3 because it does not require shipping a binary/wheel, and the CI time gains are not substantial.

I think that a build taking 4s is reasonable enough to then consider. I agree with your assessment - it avoids the hosting problem and has a negligible impact on the build.

@compnerd If you're okay with the patch, can you merge it? (I can do it, too, but I think it's better if you review it)

ping @compnerd

Yeah, I'll take a look soon, at an offsite this week.

@charles-zablit would you mind just using the same process for all architectures? That is, lets completely ignore the wheel and just build the dependencies. This makes the ARM64 and X64 paths identical and therefore less likely to be an issue when working on one architecture.

Done, this revision of the code uses pip install with the --with-hashes parameter and a pinned version to install the packages, whatever the platform is.