Add pip (with virtual environments) example for Python project #14

Surgo · 2019-10-31T15:48:11Z

This PR adds example for cache Python packages with virtual environments.

Default install path is like /opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/.
So use virtual environment to install packages into workspace and cache it.

First time (no cache)

Second time (use cache)

smorimoto · 2019-10-31T15:53:48Z

The example has been moved to examples.md. See #13.

Surgo · 2019-10-31T15:59:55Z

Moved into examples.md.

smorimoto · 2019-10-31T16:03:24Z

examples.md

-```
+```
+
+### Python - pip


Unify format.

Suggested change

### Python - pip

## Python - pip

smorimoto · 2019-10-31T16:04:57Z

Please add this to the top.

- [Python - pip](#python---pip)

smorimoto · 2019-10-31T16:06:50Z

examples.md

+- uses: actions/cache@preview
+  with:
+    path: .venv
+    key: ${{ runner.os }}-pip-${{ hashFiles(format('{0}/{1}', github.workspace, 'requirements.txt')) }}


Suggested change

key: ${{ runner.os }}-pip-${{ hashFiles(format('{0}/{1}', github.workspace, 'requirements.txt')) }}

key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}

smorimoto · 2019-10-31T16:09:02Z

examples.md

+- run: |
+    python -m venv .venv
+    source .venv/bin/activate
+    python -m pip install -r requirements.txt


Remove run section.

But .venv path for cache is created by python -m venv .venv command 🤔

This is just an example of the cache action API, not how to write a Python workflow setup.

See an example in other languages/ecosystems.

For an example of using virtualenv, changing to the minimum example code and title to Python - pip, virtualenv should enough. (imo)

Ummm... .venv is not default package install path.
Default path is/opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/ for actions/setup-python@v1 with python-version: 3.7
Is /opt/hostedtoolcache/Python/3.7.5/x64/lib/python3.7/site-packages/ better cache path in examples?

See the comment above.

smorimoto · 2019-10-31T16:23:03Z

Here is correct format.

key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}

smorimoto · 2019-10-31T16:25:59Z

examples.md

 - [Swift, Objective-C - Carthage](#swift-objective-c---carthage)
 - [Swift, Objective-C - CocoaPods](#swift-objective-c---cocoapods)
 - [Ruby - Gem](#ruby---gem)
+- [Python - pip, virtuelenv](#python---pip,-virtuelenv)


Suggested change

- [Python - pip, virtuelenv](#python---pip,-virtuelenv)

- [Python - pip, virtuelenv](#python---pip-virtuelenv)

smorimoto · 2019-10-31T16:27:53Z

If you correct the title link, overall it seems good!

smorimoto · 2019-10-31T16:29:43Z

Great! Looks good to me!

stefanvanburen · 2019-10-31T16:46:36Z

examples.md

 - [Swift, Objective-C - Carthage](#swift-objective-c---carthage)
 - [Swift, Objective-C - CocoaPods](#swift-objective-c---cocoapods)
 - [Ruby - Gem](#ruby---gem)
+- [Python - pip, virtuelenv](#python---pip-virtuelenv)


Suggested change

- [Python - pip, virtuelenv](#python---pip-virtuelenv)

- [Python - pip, virtualenv](#python---pip-virtualenv)

Good catch.

philipslan · 2019-10-31T16:54:45Z

One thing to note about caching the virtualenv is that users will have to manually activate the environment for each task. That might be worth adding.

Surgo · 2019-11-04T10:00:27Z

Cache download cache instead of virtualenv like Travis-CI

hugovk · 2019-11-04T10:04:15Z

examples.md

+```yaml
+- uses: actions/cache@preview
+  with:
+    path: ~/.cache/pip


This path only works for Linux, see #3 (comment).

Oh.. thank you for your advice :)

philipslan · 2019-11-04T19:51:05Z

examples.md

+    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
+    restore-keys: |
+      ${{ runner.os }}-pip-
+```


There should also probably be an example for windows here. The path I believe is path: ~\AppData\Local\pip\Cache from pip docs.

Confirmed: https://github.com/hugovk/pypistats/runs/288196149

- uses: actions/cache@preview if: startsWith(runner.os, 'Windows') with: path: ~\AppData\Local\pip\Cache key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip-

joshmgross

Thanks for this!

Could you add an example for windows as well? (See @philipslan's comment)

joshmgross · 2019-11-06T21:08:28Z

With a step like:

    - name: Get pip cache
      id: pip-cache
      run: |
        python -c "from pip._internal.locations import USER_CACHE_DIR; print('::set-output name=dir::' + USER_CACHE_DIR)"

We can get the cache directory from pip, so that we don't need to hardcode it and we can work universally.

Once #39 is fixed, we can remove the extra Windows action as we'll be able to use / instead of \ inside hashFiles (that fix should be rolling out soon)

Thus the example would be something like

- name: Get pip cache
   id: pip-cache
   run: |
     python -c "from pip._internal.locations import USER_CACHE_DIR; print('::set-output name=dir::' + USER_CACHE_DIR)"

- uses: actions/cache@v1
  if: startsWith(runner.os, 'Windows')
  with:
    path: ${{ steps.pip-cache.outputs.dir }}
    key: ${{ runner.os }}-pip-${{ hashFiles('**\requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

- uses: actions/cache@v1
  if: !startsWith(runner.os, 'Windows')
  with:
    path: ${{ steps.pip-cache.outputs.dir }}
    key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

hugovk · 2019-11-06T21:52:58Z

Note the underscore before internal, and the word internal: pip has no public API.

https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program

And it did change fairly recently:

https://stackoverflow.com/a/48956368/724176

joshmgross · 2019-11-07T19:28:47Z

@hugovk Good point, we shouldn't recommend using an internal API in our docs.

@Surgo could you add a Windows example following like:

- uses: actions/cache@v1
  if: startsWith(runner.os, 'Windows')
  with:
    path: ~\AppData\Local\pip\Cache
    key: ${{ runner.os }}-pip-${{ hashFiles('**\requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-pip-

If you're busy or unable to continue this PR, please let me know and I can pick it up. Thanks again for all the work you've done!

shadowmoose · 2019-11-10T09:35:55Z

Not sure if this is useful for anyone here or at Issue #3, but I've cobbled together a cross-platform cache workflow that uses one preliminary step to find the pip library location, using built-in Python libraries & supported methods.

It seems to work on all my projects. It also avoids adding extra skipped build steps.

name: CacheTest

on:
  push

jobs:
  run-tests:
    name: ${{ matrix.os }}, ${{ matrix.python-version }}
    runs-on: ${{ matrix.os }}-latest
    strategy:
      max-parallel: 3
      matrix:
        os: [ubuntu, windows, macOS]
        python-version: [3.5, 3.6, 3.7]

    steps:
      - uses: actions/checkout@v1
      - name: Set up Python
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python-version }}
      
      - name: Get Python Location
        run: |
          echo "::set-output name=path::$(python -c 'import site; print(site.getusersitepackages())')"
        id: get_location

      - name: Activate Cache
        uses: actions/cache@preview
        with:
          path: ${{ steps.get_location.outputs.path }}
          key: pytest-${{ matrix.os }}-${{ matrix.python-version }}-pip-${{ hashFiles('**/**equirement*.txt') }}
          restore-keys: |
            pytest-${{ matrix.os }}-${{ matrix.python-version }}-pip

      - name: Install Dependencies
        run: |
          python -m pip install --upgrade pip setuptools wheel
          pip install --upgrade -r requirements.txt --user
          # (Requirements are now installed to the cached user directory.)

joshmgross · 2019-11-12T21:20:21Z

Resuming the work for PR at #86

Surgo · 2019-11-13T11:01:53Z

Sorry my late response 🙇

Surgo force-pushed the master branch from 26131a3 to 96433bb Compare October 31, 2019 15:59

smorimoto reviewed Oct 31, 2019

View reviewed changes

stefanvanburen reviewed Oct 31, 2019

View reviewed changes

Surgo force-pushed the master branch from 55420b2 to bc20765 Compare November 2, 2019 13:59

Surgo added 6 commits November 4, 2019 19:02

Add pip (with virtual environments) example for Python project

8505450

Fix wrong heading

62dafe9

Add index

4056deb

Add wildcard

8a5e235

Add virtualenv for title and fix wrong format

f1938ac

Fix wrong link

6fac62b

hugovk reviewed Nov 4, 2019

View reviewed changes

Surgo added 3 commits November 4, 2019 19:04

Fix typo

221ee97

Add note for activate environment for each tasks

643f38f

Cache downloaded cache insted of virtualenv

315c827

Surgo force-pushed the master branch from 0236cb0 to 315c827 Compare November 4, 2019 10:06

Support macOS

786d2db

philipslan reviewed Nov 4, 2019

View reviewed changes

philipslan approved these changes Nov 5, 2019

View reviewed changes

joshmgross suggested changes Nov 5, 2019

View reviewed changes

peaceiris mentioned this pull request Nov 10, 2019

proposal: Add cache step for Python project peaceiris/actions-gh-pages#48

Closed

hugovk mentioned this pull request Nov 10, 2019

Add 'pip cache' command pypa/pip#6391

Merged

10 tasks

joshmgross mentioned this pull request Nov 12, 2019

Add pip examples #86

Merged

Surgo closed this Nov 13, 2019

	key: ${{ runner.os }}-pip-${{ hashFiles(format('{0}/{1}', github.workspace, 'requirements.txt')) }}
	key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}

	- [Python - pip, virtuelenv](#python---pip,-virtuelenv)
	- [Python - pip, virtuelenv](#python---pip-virtuelenv)

	- [Python - pip, virtuelenv](#python---pip-virtuelenv)
	- [Python - pip, virtualenv](#python---pip-virtualenv)

Add pip (with virtual environments) example for Python project #14

Add pip (with virtual environments) example for Python project #14

Uh oh!

Conversation

Surgo commented Oct 31, 2019

First time (no cache)

Second time (use cache)

Uh oh!

smorimoto commented Oct 31, 2019

Uh oh!

Surgo commented Oct 31, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smorimoto commented Oct 31, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Surgo Oct 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smorimoto Oct 31, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smorimoto commented Oct 31, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smorimoto commented Oct 31, 2019

Uh oh!

smorimoto commented Oct 31, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

philipslan commented Oct 31, 2019

Uh oh!

Surgo commented Nov 4, 2019

Uh oh!

hugovk Nov 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshmgross left a comment

Choose a reason for hiding this comment

Uh oh!

joshmgross commented Nov 6, 2019

Uh oh!

hugovk commented Nov 6, 2019

Uh oh!

joshmgross commented Nov 7, 2019

Uh oh!

shadowmoose commented Nov 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshmgross commented Nov 12, 2019

Uh oh!

Surgo commented Nov 13, 2019

Uh oh!

Surgo Oct 31, 2019 •

edited

Loading

smorimoto Oct 31, 2019 •

edited

Loading

hugovk Nov 4, 2019 •

edited

Loading

shadowmoose commented Nov 10, 2019 •

edited

Loading