51

Command:

pip install -r requirements.txt

Output:

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.

If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.

See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

I wish someone would explain to me what to do and how to solve it.

6
  • Are you sure you were not meant to do this in a virtualenv instead? Mar 1 at 10:03
  • 1
    The best way to solve this issue is to create a virtual environment with python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip to install the packages in the requirements.txt file. Make sure you have python3-full installed. Once the virtual environment is created and the packages are installed, you can then activate the virtual environment by running source path/to/venv/bin/activate. Mar 1 at 10:05
  • There is a conflict between you district package manager and pip, see into PEP668
    – Hermann12
    Mar 6 at 5:59
  • 2
    Im confused.. python3-full is different from python3 ? what is this "full"? Jun 14 at 21:11
  • pipx works nicely as well Jun 21 at 9:51

9 Answers 9

68

This is due to your distribution adopting PEP 668 – Marking Python base environments as “externally managed”.

TL;DR: Use a venv:

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -r requirements.txt

Long story: Your distribution is trying to protect you against mixing apt provided packages and pip provided packages. Mixing two package managers (apt and pip here) is always a bad idea and the source of many issues.

PEP 668 is a way for distributions to explicitly tell users to avoid falling into this pitfall. Your distribution told you three solutions in the message, but only the 2nd one applies cleanly to your use case:

  • Using apt install python3-xxx. It does not cleanly apply for you as you're having a requirements.txt, not a single dependency. It would work if you have only a few requirements in the file and can do it manually for each, like apt install python3-xxx python3-yyy python3-zzz. In this case there's no weird mixing of package managers: you installed python3 using apt, you're installing your dependencies using apt: no surprises.
  • Using a venv: python3 -m venv .venv then source .venv/bin/activate, then pip install -r requirements.txt. In this case the installation is contained inside the .venv directory: no mixing of what apt does and what pip does, no surprises.
  • Using pipx which does not cleanly apply to your case, pipx is good to install and use a program, like pipx install black, but here you need to install libraries listed in a requirement file, not a program.

There's another way, that I often use myself because I often need multiple different Python versions:

  • Use a Python not provided by your distribution, so it does not mess with apt installed things and does not adopt PEP 668. I often compile many Python interpreters myself using a short bash function which use a --prefix=~/.local/, for testing purposes. With those Pythons I use either a venv either a good old pip install, in this case pip will install it in ~/.local/, again no clash between apt and pip, no bad surprises.
5
  • 5
    I just ran into this after upgrading to Debian 12. I understand why it's a good practice but I won't say it isn't annoying. Jun 20 at 22:14
  • 1
    I need to run sudo apt install python3-venv frist.
    – rayzinnz
    Jul 2 at 5:24
  • 1
    Or sudo apt install python3-full. But yes, you need it, and it's 100% OK to sudo apt install it, that's the right way. Jul 3 at 8:05
  • 4
    source is a bash extension. . will work in all shells.
    – mernst
    Jul 20 at 14:39
  • 1
    "I understand why it's a good practice but I won't say it isn't annoying." - Yea ... it is annoying to be told to do it the right way :-)
    – Stephen C
    Aug 16 at 1:19
21

Use --break-system-packages at the end of pip.

This will install packages in your local user directory, ~/.local/lib/python3.11.

pip install xyz --break-system-packages
5
  • 2
    For projects it will be recommented to use venv May 12 at 9:37
  • 1
    This may get rid of the symptom, but what is the risk? "--break-system-packages" sounds scary. Sep 7 at 9:15
  • From another answer: "Your distribution is trying to protect you against mixing apt provided packages and pip provided packages. Mixing two package managers (apt and pip here) is always a bad idea" Sep 7 at 10:05
  • Though it did make QMK install on Ubuntu 23.04 (Lunar Lobster): python3 -m pip install --break-system-packages --user qmk. But the consequences or risks are unknown. Sep 7 at 19:21
  • 3
    This is perfect when creating a linux docker container running a python application. It makes no sense creating a venv inside a container.
    – robsn
    Oct 24 at 8:19
7

Add this to your ~/.config/pip/pip.conf file:

[global]
break-system-packages = true

If you want to know more, you can click here.

3
  • 2
    This should be the correct answer here. The question is about solving the externally managed Python error. Forcing everyone to use a new approach to Python isn't a solution, it's a philosophical position. I really didn't expect it to be this simple. I was expecting your answer not to work but you really saved my life here. This must be the single most idiotic change the Python devs have made to the project in recent years. I'm now going to have to memorize this as it completely breaks all the packages on VS Code. Aug 28 at 0:59
  • This may get rid of the symptom, but what is the risk? "--break-system-packages" sounds scary. Can you provide an explanation, please? Sep 7 at 9:38
  • From another answer: "Your distribution is trying to protect you against mixing apt provided packages and pip provided packages. Mixing two package managers (apt and pip here) is always a bad idea" Sep 7 at 10:05
2

I was faced with this error inside pyenv after upgrading to Ubuntu 23.04 (Lunar Lobster). As part of this upgrade, the system Python was also upgraded from 3.10 to 3.11. Re-installing pyenv didn't help. I had to:

  1. remove all instances of pip discovered via which pip (and also sudo which pip as it kept pointing me to .pyenv/shims and root user didn't have pyenv installed)
  2. switch to any pyenv installed python with pyenv global <version>
  3. install pip back with get-pip.py:
    wget https://bootstrap.pypa.io/get-pip.py
    python get-pip.py
    rm get-pip.py
    
0

Use:

sudo rm /usr/lib/python3.11/EXTERNALLY-MANAGED
2
  • 3
    Answer needs supporting information Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – moken
    Aug 3 at 10:01
  • On what system was this tested on? Sep 7 at 9:19
0
sudo rm /usr/lib/python3.*/EXTERNALLY-MANAGED
0

While accepted answer is correct for regular Python usage, I miss the part about Docker. As nicely explained in this answer, many people didn't use venv in Docker because there is no really value in it, because containers by definition provide an isolated environment.

As even mentioned in the PEP-668 itself, it might be okay to remove the file marker:

So, builders of base container images may want to ensure that the marker file is not present, even if the underlying OS ships one by default.

However, I feel like using venv would be still a better approach, and if one needs a change in the Dockerfile anyway, why not do it proper way?

# Assuming somewhere on top there is already `python3-venv` installed 
# via system package manager, e.g. `apt-get install -y python3-venv`

RUN python3 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Below you can continue using `pip install` as normal
0

For the people coming here to fix this problem because of installed python version with brew (in my case with python3.12), assuming your homebrew installation is in /opt/homebrew:

# remove all externally managed files for all python binaries in brew

rm /opt/homebrew/Cellar/python\@3*/**/EXTERNALLY-MANAGED
-2

The message already passes the solution:

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

Try this:

sudo apt install python3-<package>

It worked for me.

2
  • 1
    Unfortunately that did not work for me. I think what happened was I ran get-pip.py, then tried apt install python3-pip, which broke my configs
    – Alice
    May 20 at 19:02
  • Well, what if people have requirements.txt and cannot really do apt install for every single line of requirements? Nov 9 at 12:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.