Problem with config cache of WASI #118039

nineteendo · 2024-04-18T11:48:17Z

This workflow on #117855 keeps failing. It asks to delete the cache, which I obviously can't do. How should this be fixed? Prune the cache or update the workflow?

Run python3 Tools/wasm/wasi.py configure-build-python -- --config-cache --with-pydebug
  python3 Tools/wasm/wasi.py configure-build-python -- --config-cache --with-pydebug
  shell: /usr/bin/bash -e {0}
  env:
    WASMTIME_VERSION: 18.0.3
    WASI_SDK_VERSION: [2](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:2)1
    WASI_SDK_PATH: /opt/wasi-sdk
    CROSS_BUILD_PYTHON: cross-build/build
    CROSS_BUILD_WASI: cross-build/wasm[3](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:3)2-wasi
    PATH: /usr/lib/ccache:/opt/hostedtoolcache/wasmtime/18.0.3/x6[4](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:4):/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
    pythonLocation: /opt/hostedtoolcache/Python/3.12.3/x[6](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:6)4
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.12.3/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.3/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.3/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.3/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.3/x64/lib
tput: No value for $TERM and no -T specified
configure: loading cache config.cache
configure: error: `PKG_CONFIG_PATH' has changed since the previous run:
configure:   former value:  `/opt/hostedtoolcache/Python/3.12.2/x64/lib/pkgconfig'
configure:   current value: `/opt/hostedtoolcache/Python/3.12.3/x64/lib/pkgconfig'
configure: error: in `/home/runner/work/cpython/cpython/cross-build/build':
configure: error: changes in the environment can compromise the build
configure: error: run `make distclean' and/or `rm config.cache'
	    and start over
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Tools/wasm/wasi.py", line 34[7](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:7), in <module>
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
    main()
  File "/home/runner/work/cpython/cpython/Tools/wasm/wasi.py", line 343, in main
    dispatch[context.subcommand](context)
  File "/home/runner/work/cpython/cpython/Tools/wasm/wasi.py", line 7[9](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:9), in wrapper
    return func(context, working_dir)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/cpython/cpython/Tools/wasm/wasi.py", line 137, in configure_build_python
    call(configure, quiet=context.quiet)
  File "/home/runner/work/cpython/cpython/Tools/wasm/wasi.py", line [10](https://github.com/python/cpython/actions/runs/8733446361/job/23962151465?pr=117855#step:10:10)3, in call
    subprocess.check_call(command, **kwargs, stdout=stdout, stderr=stderr)
  File "/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/subprocess.py", line 413, in check_call
📁 /home/runner/work/cpython/cpython/cross-build/build
📝 Touching /home/runner/work/cpython/cpython/Modules/Setup.local ...
❯ ../../configure --config-cache --with-pydebug
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['../../configure', '--config-cache', '--with-pydebug']' returned non-zero exit status 1.
Error: Process completed with exit code 1

Linked PRs

gh-118039: Fix config.cache key on WASI #118137

The text was updated successfully, but these errors were encountered:

hugovk · 2024-04-18T12:56:53Z

The error is:

configure: loading cache config.cache
configure: error: `PKG_CONFIG_PATH' has changed since the previous run:
configure:   former value:  `/opt/hostedtoolcache/Python/3.12.2/x64/lib/pkgconfig'
configure:   current value: `/opt/hostedtoolcache/Python/3.12.3/x64/lib/pkgconfig'
configure: error: in `/home/runner/work/cpython/cpython/cross-build/build':
configure: error: changes in the environment can compromise the build
configure: error: run `make distclean' and/or `rm config.cache'
	    and start over

The problem is the config.cache doesn't match the new Python patch version (provided by actions/setup-python).

It's failing at:

cpython/.github/workflows/reusable-wasi.yml

Lines 54 to 55 in 0a0756c

    
           - name: "Configure build Python" 
        
             run: python3 Tools/wasm/wasi.py configure-build-python -- --config-cache --with-pydebug

The previous step is restoring the cache from GitHub's actions/cache:

cpython/.github/workflows/reusable-wasi.yml

Lines 49 to 53 in 0a0756c

    
           - name: "Restore Python build config.cache" 
        
             uses: actions/cache@v4 
        
             with: 
        
               path: ${{ env.CROSS_BUILD_PYTHON }}/config.cache 
        
               key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}

We can see the key: line. This takes those values to figure out the cache's key. If it can find a cache file matching this, it will restore it. So for example, if the env.IMAGE_VERSION changes (iirc this is the GitHub Actions image version), then we get a new value and it won't restore an old cache, but will save a new one for next time.

We should find a way to get the Python x.y.z version and add it in there as a key.

And it would be worth checking other workflows in case they run into something similar.

nineteendo · 2024-04-20T12:31:54Z

Can we purge the cache, or do we need it to test potential solutions? My pull request can't be merged until that test suceeds.
Should I create a new one if this takes too long?

hugovk · 2024-04-21T18:58:41Z

Yeah, as mentioned, it would be good to keep it to test potential solutions.

Would you like to look into a fix? Or I can check it next week.

nineteendo · 2024-04-21T19:08:01Z

it would be good to keep it to test potential solutions.

Now that I think about it, any changes to the key would make the test pass, regardless of whether the issue has been fixed. e.g.

key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}-dummy

So, I don't think it's useful to keep the cache.

Would you like to look into a fix?

Yeah, but as previously stated, testing it will be difficult.

nineteendo · 2024-04-21T19:23:49Z

Do you think this would work? env.pythonLocation is already used for build.yml:

-key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
+key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}-${{ env.pythonLocation }}

hugovk · 2024-04-21T19:37:38Z

Now that I think about it, any changes to the key would make the test pass, regardless of whether the issue has been fixed. e.g.
key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}-dummy

Ah, yes.

Do you think this would work? env.pythonLocation is already used for build.yml:

-key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}
+key: ${{ github.job }}-${{ runner.os }}-${{ env.IMAGE_VERSION }}-${{ inputs.config_hash }}-${{ env.pythonLocation }}

For example:

pythonLocation: /opt/hostedtoolcache/Python/3.12.3/x64

Yep, that could do it!

Looking at https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#outputs-and-environment-variables there's also things like steps.my-step-id.outputs.python-version but env.pythonLocation looks cleaner.

nineteendo · 2024-04-21T19:42:40Z

It indeed seems to be working:

Cache not found for input keys: build_wasi_reusable-Linux--24168b7150a51bf2d84cf11b0c1766713f3008fec4733f0682c1e541e1764fc1-/opt/hostedtoolcache/Python/3.12.3/x64

nineteendo · 2024-04-21T19:44:16Z

I'll make a pull request.

hugovk · 2024-04-23T07:50:39Z

Thanks for the PR!

Eclips4 added the OS-wasi label Apr 18, 2024

nineteendo mentioned this issue Apr 18, 2024

gh-117587: Add C implementation of os.path.abspath #117855

Closed

This comment was marked as resolved.

Sign in to view

bedevere-app bot mentioned this issue Apr 21, 2024

gh-118039: Fix config.cache key on WASI #118137

Merged

hugovk pushed a commit that referenced this issue Apr 23, 2024

gh-118039: Fix config.cache key on WASI (#118137)

456c29c

hugovk closed this as completed Apr 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Problem with config cache of WASI #118039

Problem with config cache of WASI #118039

nineteendo commented Apr 18, 2024 •

edited by bedevere-app bot

Loading

hugovk commented Apr 18, 2024 •

edited

Loading

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

nineteendo commented Apr 20, 2024

Uh oh!

hugovk commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

hugovk commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

hugovk commented Apr 23, 2024

Uh oh!

Uh oh!

Problem with config cache of WASI #118039

Problem with config cache of WASI #118039

Comments

nineteendo commented Apr 18, 2024 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked PRs

hugovk commented Apr 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

nineteendo commented Apr 20, 2024

Uh oh!

hugovk commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

hugovk commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

nineteendo commented Apr 21, 2024

Uh oh!

hugovk commented Apr 23, 2024

Uh oh!

nineteendo commented Apr 18, 2024 •

edited by bedevere-app bot

Loading

hugovk commented Apr 18, 2024 •

edited

Loading