Thanks to visit codestin.com
Credit goes to github.com

Skip to content

PyPy3 wheels not uploaded to cache -> very slow builds #288

Closed
@hugovk

Description

@hugovk

Description:

Wheels built by PyPy3 aren't being cached. This makes the PyPy3 build very slow as it rebuilds them every time.

Action version:

v2 = v2.3.1

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:

PyPy 7.3.7 with Python (3.8.12)

(Also Python 3.7-3.10 in the matrix)

Repro steps:

I'm not entirely sure how the current state came about but:

  1. Trigger a build
  2. The existing pip cache is downloaded, but it doesn't contain wheels for pypy-3.8
  3. requirements.txt is installed. Some pure Python wheels are installed from cache, but a number of sdists are downloaded: matplotlib-3.5.0.tar.gz, kiwisolver-1.3.2.tar.gz, Pillow-8.4.0.tar.gz
  4. Wheels are built for these sdists. It is takes nearly 9 minutes to install everything.

Expected behavior:

I expect after the wheels are build the first time, the pip cache is uploaded with actions/cache for next time.

Actual behavior:

The pip cache is never uploaded for PyPy3.

"Post Set up Python pypy-3.8" shows either:

Post job cleanup.
Unable to reserve cache with key setup-python-Linux-pip-4547a7629556c7c4fb4f7b36b381999277204f48e1d116d1dcdc783c98b8aa0c, another job may be creating this cache.

https://github.com/hugovk/drop-python/runs/4384155862?check_suite_focus=true

Or:

Post job cleanup.
Cache hit occurred on the primary key setup-python-Linux-pip-f5b11c7530ca1ca017e33203e65a953e8864e5891c8f500684b94ffcdde1d825, not saving cache.

https://github.com/hugovk/drop-python/runs/4383955419?check_suite_focus=true


I see the cache key is calculated from the runner OS (e.g. Linux), the package manager (e.g. pip or pipenv) and the hash of the dependency path:

const hash = await glob.hashFiles(this.cacheDependencyPath);
const primaryKey = `${this.CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${this.packageManager}-${hash}`;
const restoreKey = `${this.CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${this.packageManager}`;

For a given OS, pip should be able to handle caching different Python versions in the same cache dir. However, there seems to be some sort of race condition here.

Before using this new caching I used to include the Python version in the cache key:

          key:
            ${{ matrix.os }}-${{ matrix.python-version }}-v1-${{
            hashFiles('**/setup.py') }}
          restore-keys: |
            ${{ matrix.os }}-${{ matrix.python-version }}-v1-

Perhaps this would help here too?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions