5

I have the following steps in a github action:

steps:
      - name: Check out repository code
        uses: actions/checkout@v2

      - name: Cache dependencies
        id: pip-cache
        uses: actions/cache@v2
        with:
          path: ~.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-

      - name: Install dependencies
        if: steps.pip-cache.outputs.cache-hit != 'true'
        run: pip install -r requirements.txt
     
      - name: run mypy
        run: mypy .

The caching works fine, but when a cache hit occurs, and I try to run mypy, it fails with:

Run mypy .
/home/runner/work/_temp/9887df5b-d5cc-46d7-90e1-b884d8c49272.sh: line 1: mypy: command not found
Error: Process completed with exit code 127.

The whole point of caching dependencies is so I don't have to install them every time I run the workflow. How do I use the cached dependencies?

1 Answer 1

8

You're only caching source tarballs and binary wheels downloaded by pip. You're not caching:

  • Installed Python packages (i.e., the site-packages/ subdirectory of the active Python interpreter).
  • Installed entry points (i.e., executable commands residing in the current ${PATH}).

That isn't necessarily a bad thing. Merely downloading assets tends to consume a disproportionate share of scarce GitHub Actions (GA) minutes; caching assets trivially alleviates that issue.

In other words, remove the if: steps.pip-cache.outputs.cache-hit != 'true' line to restore your GitHub Actions (GA) workflow to sanity.

But... I Want to Cache Installed Packages!

Challenge accepted. This is feasible – albeit more fragile. I'd advise just caching pip downloads unless you've profiled the pip install command to be a significant installation bottleneck.

Let's assume that you still want to do this. In this case, something resembling the following snippet should get you where you want to go:

  - uses: 'actions/setup-python@v2'
    with:
      # CAUTION: Replace this hardcoded "3.7" string with the
      # major and minor version of your desired Python interpreter.
      python-version: "3.7"
  - uses: 'actions/cache@v2'
    id: cache
    with:
      # CAUTION: Replace this hardcoded "python3.7" dirname with
      # the dirname providing your desired Python interpreter.
      path: ${{ env.pythonLocation }}/lib/python3.7/site-packages/*
      key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
      restore-keys: |
        ${{ runner.os }}-pip-
        ${{ runner.os }}-

As noted, you'll want to manually replace the hardcoded 3.7 and python3.7 substrings above with something specific to your use case. That's one of several reasons why this is dangerously fragile. Another is that the ${{ env.pythonLocation }} environment variable set by the setup-python GitHub Action has been infamously undocumented for several years. </gulp> (env.pythonLocation documentation was added from v4.0.0.)

In theory, adding the above directly under your existing uses: actions/cache@v2 list item should suffice. Good luck and Godspeed as you travel into the terrifying unknown.

1
  • Thanks for the reply! Yes I noticed shortly after posting this that removing the if: steps.pip-cache.outputs.cache-hit != 'true' fixes the issue. I added it because I saw that the pip output was saying installing dependencies, but didn't look closely enough to see it was saying installing cached dependencies... always read your logs closely folks.
    – rcb
    Commented Nov 29, 2021 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.