94

For developing a script that runs pip install it would be useful to have a --dry-run function.

I came across the --no-install option. But this one is deprecated and on-call references this. There are hints to unpack a package only, but I can't find a unpack option in the pip documentation.

2

7 Answers 7

42

Yes - pip should have a dry-run option, to indicate what would happen in a complex situation. It is dangerous when running pip install downgrades packages without asking you. We need some way to ask what would happen if we run pip install -r requirements.txt without laboriously searching thru all the requirements and comparing them to the currently installed ones.

It looks like setup.py used to have a dry-run. Folks are asking for it elsewhere.

Some progress in that direction can be found here:

Update: Activity continues on the official bug related to this: Add `pip install --dry-run` or similar, to get resolution result · Issue #53 · pypa/pip. It is blocked on the development of the dependency resolver, which is nearing completion

1
  • 1
    an alternative to the --dry-run would be to print what would be done first then force the user interactively to acknowledge yes or no. Sep 17, 2021 at 13:22
24

Eventually it was approved and released in 22.2 version

According to the documentation, --dry-run:

Don’t actually install anything, just print what would be. Can be used in combination with --ignore-installed to ‘resolve’ the requirements.

For example:

pip install flask --dry-run

prints information about what packages would be installed without installing them (but downloading):

Collecting flask
  Using cached Flask-2.2.2-py3-none-any.whl (101 kB)
Collecting Jinja2>=3.0
  Using cached Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting Werkzeug>=2.2.2
  Using cached Werkzeug-2.2.2-py3-none-any.whl (232 kB)
Collecting click>=8.0
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting itsdangerous>=2.0
  Using cached itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting MarkupSafe>=2.0
  Using cached MarkupSafe-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Would install Flask-2.2.2 Jinja2-3.1.2 MarkupSafe-2.1.1 Werkzeug-2.2.2 click-8.1.3 itsdangerous-2.1.2

There are a few other options that were added recently (they're still experimental, pip gives a warning):

Tell me what pip installed:

pip install flask --report /tmp/report.json

Tell me what pip would install:

pip install flask --dry-run --report /tmp/report.json

Tell me what pip would install in an empty environment (aka resolve the requirements):

pip install flask --dry-run --ignore-installed --report /tmp/report.json
1
  • That --dry-run is good but not perfect. I've just tried it in a fresh venv with Python 3.10.7 to dry run soft-dtw. Most requirements were reported only but Cython and numpy were installed. Oct 12, 2022 at 13:23
18

It appears you are right, it has been deprecated (ref).

If by trial run you mean testing it out before actually installing a package in a certain place, presumably before a system wide install, then you can simply run it sandboxed using a virtual environment and then simply discard the environment.

virtualenv /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv 

Not as succinct as using a dry-run argument to pip, but it does the job. Also if you want to do a dry run of a series of package installations, omit the deletion at the end.

In a script you can distil it into a procedure:

#!/bin/bash

TMP_DIR='/tmp/venv'

function dry_run (){
    if [ ! -d "$TMP_DIR" ]; then
            virtualenv /tmp/venv
    fi
    /tmp/venv/bin/pip install $1
}

dry_run flask
dry_run uwsgi
rm -rf $TMP_DIR

If you want to do a dry run that tests that the new install(s) play well with system wide deployed, then use virtualenv's system-site-packages option.

virtualenv --system-site-packages /tmp/venv; /tmp/venv/bin/pip install flask; rm -rf /tmp/venv
2
  • It has issues for cases where /tmp is mounted noexec (Which it should be - it can really stop a few exploits...) Mar 27, 2018 at 10:03
  • 1
    Nice hack to check dependencies or populate pip's cache, but it doesn't show what pip would do to an existing environment (be it virtual, conda, system or user). Jul 3, 2020 at 6:24
12

The pip-sync command from pip-tools can report what packages would be installed, but will also output the ones that are installed but not in the requirements file. Its dry run option is -n

$ pip install pip-tools
$ pip-sync -n requirements.txt 
Would uninstall:
  pip-tools
Would install:
  requests

Here's the help from pip-sync:

pip-sync --help
Usage: pip-sync [OPTIONS] [SRC_FILES]...

  Synchronize virtual environment with requirements.txt.

Options:
  --version               Show the version and exit.
  -n, --dry-run           Only show what would happen, don't change anything
  --force                 Proceed even if conflicts are found
  -f, --find-links TEXT   Look for archives in this directory or on this HTML
                          page
  -i, --index-url TEXT    Change index URL (defaults to PyPI)
  --extra-index-url TEXT  Add additional index URL to search
  --trusted-host TEXT     Mark this host as trusted, even though it does not
                          have valid or any HTTPS.
  --no-index              Ignore package index (only looking at --find-links
                          URLs instead)
  -q, --quiet             Give less output
  --user                  Restrict attention to user directory
  --cert TEXT             Path to alternate CA bundle.
  --client-cert TEXT      Path to SSL client certificate, a single file
                          containing the private key and the certificate in
                          PEM format.
  --help                  Show this message and exit.
3
3

This approach uses the current environment you are using and overrides paths pythons sees, it uses a similar approach to pipenv and virtualenv, but without any additional env creation, no changes to the current env, self-cleaning, and taking your current installed packages in the final dependency list solving algorythm, you know, just like a proper dry run.

  1. pip has a --target flag that lets you point to some arbitrary directory to install packages to, instead of system packages
  2. add that path to your PYTHONPATH temporarily, so you can figure out what was installed
  3. if it fails, well, then it failed, otherwise you can just generate a freeze list
  4. finally, a sweet diff for your cognitive convenience

So, the script would be something like this:

export DRY_RUN=/tmp/python-dry-run #points to the temp dir
mkdir -p $DRY_RUN #creates the dir
pip install -r requirements.txt --target $DRY_RUN #dry run happens here
PYTHONPATH=$DRY_RUN pip freeze > requirements-dry-run.txt #dry run output
diff --color requirements.txt requirements-dry-run.txt #red removed, green added
rm -rf $DRY_RUN #cleanup

caveats, warnings and opinions:

  1. try not to export PYTHONPATH directly in your shell, that might make things confusing, instead pass it as a temp variable at the beginning of the command, just like I did there

  2. pip freeze sucks, it usually leads to dependencies getting deadlocked in the future, so try your best to maintain your requirements file manually

  3. if you're asking for this, you're probably in a dependency hell. An easy way of getting rid of requirements.txt deadlock nightmare is to just to get rid of mid and minor versions completely and letting pip figure out the dependencies with a bit more leeway. So, if you are using example==1.4.5 in your requirements.txt, make it example>=1

2

[Ugly hack disclaimer] on Linux you can try to install in the system location as a user who does not have permission to install in the /usr/ directory (pip3 install --system on Debian/Ubuntu/etc.) The command fails with "Permission denied" but only after logging what is missing and what is not.

Makes you wonder how hard an actual dry-run option would really be to implement?

3
  • But this will already download all packages before it fails to install
    – rubo77
    Jul 7, 2021 at 0:09
  • 1
    Correct. Downloading packages has no side effect and never bothered me.
    – MarcH
    Jul 8, 2021 at 1:04
  • 1
    Newer pip versions will automatically do a user install... (an possibly unistall user packages to make room for installed versions... Which is the main thing we want to prevent with a --dry-run) Oct 15, 2021 at 15:20
-2

With pip version 9 there's a new option --format freeze leading to an elegant one line solution for the pip install -r use case:

pip list --format freeze | diff - requirements.txt
2
  • 3
    For anything but the most trival cases this will show a massive difference as requirements.txt won't include the transitive dependencies (dependencies of dependencies) Jun 21, 2017 at 9:18
  • 1
    You will actually get a pretty good quick-and-dirty result by doing something like pip freeze | grep -f requirements.txt | sort | diff -y --suppress-common-lines - <(sort requirements.txt)
    – Claude
    May 19, 2018 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.