158

I have a new Macbook - a user installed it, and then I installed a new user (mine), granted admin privileges and deleted the old one. I am on OS Catalina.

Since the installation I've been having several permission problems. VSCode can't find Jupyter Notebook, pip installs packages at ~/Library/Python/3.7/site-packages.

When I do which python3 I get usr/bin/python3. When I do pip3 install <package> I get: Defaulting to user installation because normal site-packages is not writeable And then it says it has already been installed, even though I can't access it when I do import <package>.

It's seems clear that this is a permission problem, pip can't install to the "base" python, and them python can't find what I've installed into ~/Library/Python/3.7/site-packages.

I've tried reinstalling the OS, but since I haven't done a clean install, it didn't change anything. What am I missing? How exactly can I fix permissions? Where do I want packages to be installed (venv sure, but some packages I want global (like jupyter).

4
  • 1
    I'm having this problem as well. I wonder if it has to do with the end of Python 2.7 support Commented Feb 8, 2020 at 18:42
  • 7
    Are you sure that you are using the correct pip? What if you use the much safer python -m pip install ....? Commented Feb 10, 2020 at 14:10
  • 5
    @TomdeGeus: Well, python3 -m pip install ... in this case, but yes, when there are multiple versions of Python involved, it's much safer to invoke pip via the "exec module" approach, so you know you're using the expected Python executable. Commented Feb 18, 2020 at 14:46
  • 2
    To fix this I ended up uninstalling python3 (previously installed with brew) and using pyenv to install python3. Now it all works. Followed this guide: opensource.com/article/19/5/python-3-default-mac (skip to last section) Commented Mar 19, 2020 at 15:36

18 Answers 18

68

As @TomdeGeus mentioned in the comments, this command works for me:

Python 3:

python3 -m pip install [package_name]

Python 2:

python -m pip install [package_name]
4
  • 61
    I have this problem even though I am using the command from this answer
    – Jan Pisl
    Commented Aug 25, 2021 at 11:25
  • 1
    Just my 2 cents for those on work/school laptops - make sure you have write access to the python install directory (in case your admin has locked C:\ProgramFiles\Python36 for example)....
    – Salvatore
    Commented Jan 4, 2023 at 1:11
  • This is elegant: Let the python installation you want to enhance execute the pip. This worked for me on Catalina using python3.9 installed by mac ports. Commented Mar 18, 2023 at 21:40
  • I also had this problem even with adding python3. Recreating virtual environment helped. Commented Jun 29, 2023 at 20:28
33

python3.7 -m pip install [package_name]

(you should use the version that you have, of course)

solved it for me.

The most voted answer python3 -m pip install [package_name] does not help me here.

In my case, this was caused by a conflict with the dominating 3.6 version that was also installed as a default. You might ask yourself why you have 3.6 on your system, you will most probably not use that version now. The reason is that 3.6 is used as an independent default python version for many package installers. Those installers do not want to check which individual version you use and whether that fits, they just use 3.6 as a default, if you like it or not.

Here is a proof by example --upgrade pip:

pip3 install --upgrade pip

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /home/USERNAME/.local/lib/python3.6/site-packages (20.3.1)

python3 -m pip install --upgrade pip

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /home/USERNAME/.local/lib/python3.6/site-packages (20.3.1)

python3.7 -m pip install --upgrade pip

Collecting pip
Cache entry deserialization failed, entry ignored
Using cached https://files.pythonhosted.org/packages/ab/11/2dc62c5263d9eb322f2f028f7b56cd9d096bb8988fcf82d65fa2e4057afe/pip-20.3.1-py2.py3-none-any.whl
Installing collected packages: pip Successfully installed pip-20.3.1

3
  • 4
    Yep... python3 was problematic but switching to invoking python3.9 resolved things. Painful...
    – dat
    Commented Nov 16, 2021 at 5:38
  • 14
    This doesn't work for me with python3.10, unfortunately.
    – greenbug
    Commented Nov 11, 2022 at 16:54
  • None of these variations work in Monterey.
    – John Smith
    Commented Apr 13 at 4:58
31

It's best to not use the system-provided Python directly. Leave that one alone since the OS can change it in undesired ways, as you experienced.

The best practice is to configure your own Python version(s) and manage them on a per-project basis using virtualenv (for Python 2) or venv, possibly via poetry, (for Python 3). This eliminates all dependency on the system-provided Python version, and also isolates each project from other projects on the machine.

Each project can have a different Python point version if needed, and gets its own site_packages directory so pip-installed libraries can also have different versions by project. This approach is a major problem-avoider.

4
  • 3
    I had the same problem and I also had venv set. What I had to do was to execute the pip within my virtual environment "venv" as follows: .venv/bin/pip install -r pypackages.txt Commented Sep 15, 2020 at 15:12
  • 10
    @chris-johnson: Everything you said makes sense, but how do I do it exactly? On my new MBP M1, when I tried, in venv, to do pip3 install gjango, I got this error message: Defaulting to user installation because normal site-packages is not writeable ERROR: Could not find a version that satisfies the requirement gjango (from versions: none) ERROR: No matching distribution found for gjango
    – YCode
    Commented Dec 21, 2021 at 20:02
  • 2
    Downvoted. Without clear instructions, this comment doesn't do anything for me.
    – John Smith
    Commented Apr 13 at 4:59
  • @JohnSmith moreover, I have this problem with a venv. Commented Jun 3 at 12:43
15

I'm using Anaconda on Ubuntu and had the same problem.I fixed it by the following steps:

deactivating current environment

conda deactivate

Then, the base environment activates. I deactivated the base conda environment too. To do so, I used conda deactivate again.

Finally, I activate my project environment directly (instead of activating from the base environment) by the following command. Afterward, I installed the intended package successfully and worked perfectly.

conda activate myenv
pip install somepackage
2
  • 1
    This works like a charm. I wonder why this works thought. Commented Aug 17, 2023 at 9:50
  • After running this, I still get the message that it's using the cached package
    – Adam_G
    Commented Feb 4 at 16:49
9

sudo pip install

Worked for me. But pip install is not recommended to be run with sudo. The issue I was facing on BIGSUR was, it was using system python. Once I Installed python 3.9 using

brew install [email protected]

Then pip worked fine

5

For me, none of the suggestions worked so I had to delete the current virtual environment folder venv and recreate it using one of the following commands:

python -m venv venv
python3 -m venv venv

Check the source of pip on Ubuntu 20.04

which pip

returns the correct path

/home/myname/fullstack/person_api/venv/bin/pip

UPDATE 2

I suggest using pyenv to switch between different python versions and better manage python path, installations. For example:

pyenv install 3.10.4
pyenv global 3.10.4

This could save you a lot of troubles in the future.

UPDATE 1

I presume that some might encounter this problem because they set python path as environmental variable like this in ~/.bashrc:

python=/path/to/python

which you should not be doing! Instead we could do:

py=python
PATH=/path/to/python:$PATH

I bumped into this issue specifically because of this!

2
  • 1
    Solved my issue with Defaulting to user installation because normal site-packages is not writeable - perfect! Thanks! :)
    – Joachim
    Commented Nov 26, 2022 at 18:22
  • I can't follow this. Boy, that page about pyenv might as well be in sanskrit. Boy is this frustrating. Python worked, now I downgraded* from Catalina to Monterey and it doesn't, and I can't get any clear instructions to get it working again. (Yes, it's a downgrade, as most newer MacOS versions for the past 10 years or so are usually downgrades from the previous version.)
    – John Smith
    Commented Apr 13 at 5:02
2

It occurs with me when I the virtual enviroment folder name was : venv.

in this case, It gives errors like :

No module pip

Default folder is unwritable

renaming the folder solve the proplem.

1
  • worked flawlessly for me when the changed the command topython -m venv my-venv
    – LionsDen
    Commented Dec 26, 2020 at 8:54
2

I tried ever single recommendation described here. In every instance, I get the exact same result: SyntaxError: invalid syntax (<stdin>, line 1)

I'm not sure who designed the system like this, but it seems basically useless, based on my experience so far. Either create a system that works, or don't create anything at all.

2
  • I switched to Amazon SageMaker, started a Jupyter Notebook there, and everything ran fine on the very first try!
    – ASH
    Commented Aug 26, 2022 at 20:56
  • I'm with you. This is massively frustrating. Especially because it used to just work and now I'm spending hours running in circles for no productive result.
    – John Smith
    Commented Apr 13 at 5:08
2

Check on the command line "which python" to see if it is the value you expect. If you have a virtual environment activated, check /venv/bin/activate to see the value of VIRTUAL_ENV= and make sure it is the correct path . The path may be wrong if you renamed or moved the project. If the path is wrong, you can delete the venv and make a new one.

1
  • I had exactly that : i activated the venv, then moved its directory, after that pip was not detecting the venv, and trying to install at the system level instead. I just deactivate then source path-to-venv/bin/activate and it worked again.
    – Lenormju
    Commented Nov 9, 2022 at 7:26
1

Had this same issue on a fresh install of Debian 9.12. Rebooting my server solved the issue.

1

In my case on Linux, the ownership of the conda env directory had changed to another Linux user (long story), and so the the normal site-packages was not writeable due to a permissions issue.

The solution was to change ownership back to the user doing pip install.

1

Had similar issue on Ubuntu 20.04.4 LTS in VirtualBox, but none of the suggestions here worked for me.

I was trying to install open3d in a venv and every time I was getting "Defaulting to user installation because normal site-packages is not writeable" which at first I didn't even noticed. open3d was always being installed in /usr/bin/python3 environment. I've restarted the VM but without luck, so I guess the problem was not just missing write access.

So in VS Code, which was using the venv, importing open3d was not possible. But testing from terminal from the activated venv with python3 -c "import open3d as o3d; print(o3d.__version__)" was working fine and that confused me totally. I even broke my system pip installation using sudo, see further below if you want to know how to fix it.

Anyhow, the solution to my problem was to explicitly point to the python3 file in the venv where I wanted to install the package:

venv/bin/python3 -m pip install open3d

So I was testing out everything and eventually installed with sudo: sudo pip3 install open3d. This of course didn't solved the problem and open3d was still missing in the venv. Even worse, I got the message:

"WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available. You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command."

So I did it but with sudo, updating the system pip and then found out here that this is not good:

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Following an advice here, I tried to revert to original version, only then pip3 broke:

sudo pip3 uninstall pip
sudo pip3 --version
sudo: pip3: command not found

The apt package was still there:

sudo apt install python3-pip python3-pip is already the newest version (20.0.2-5ubuntu1.6).

So I had to reinstalled to fix the problem:

sudo apt-get remove python3-pip
sudo apt install python3-pip
1

For those running on a Pi, that accidentally installed pip as root. Just chown the lib folder to the pi user:

sudo chown -R pi:pi /usr/local/lib/python3.9/

0

in my case python3 -m pip install [package_name] did not solve that. in my case, it was a problem related to other processes occupying the directory. I restart Pycharm and close any other program that might occupy this folder, and reinstalled the package in site-packages directory successfully.

0

When this problem occurred to me I have tried all the mentioned approaches but they don't seem to work.

Instead, restarting Python language server in my VSCode did the job - my SimPy package is now found. On Mac it is Cmd+Shift+P and select "Python: Restart Language Server".

1
  • VSCode? What? Huh? What is VSCode?
    – John Smith
    Commented Apr 13 at 5:08
0

I met exactly the same issue.

I just type sudo python3.8 -m pip install .... and the error disappeared.

If I remove the sudo, issue remains.

0

Maybe you have python, python3, pip or pip3 aliased. In that case pip might not work well anymore, as the alias isn't always available and so pip/pip3 might resolve python/python3 differently compared to in your terminal.

That could give rise to pip/pip3 trying to install in the system python, and that could give rise to your error.

0

For readers who thought themselves accidentally update system pip:

If you saw this info in your terminal output:

Defaulting to user installation because normal site-packages is not writeable

then you will be fine. Use the pip3 you just updated to run:

pyenv global system # since I use pyenv
pip3 uninstall pip # this one does the trick

Then you can check again pip3 --version will point to the original old (XCode/System-)pip. E.g. (2022/2/28):

pip 20.2.3 from /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages/pip (python 3.8)
5
  • 5
    Could you explain please?
    – lalebarde
    Commented Oct 17, 2022 at 20:26
  • @lalebarde: In short, we naive users cannot update system-pip since its corresponding site-packages are located in a restricted folder (thus we see the not writable-message). So, instead, if you didn't see the message, you're dead. Since that would mean that you've typed in some sudo-command to unlocked the system folders restriction at some point in the past. Commented Oct 19, 2022 at 8:20
  • I call this: "if you're warned, then you're fine." when dealing with terminal stuffs :) since a well-designed program will always double-check your dangerous behaviours. Commented Oct 19, 2022 at 8:25
  • I down-voted this post. Please fix the title of this post, because it is very bad grammar. People don't "thought themselves". I will not let your sloppiness slide. I will remove my downvote after you fix it so that it explains well what your answer provides to readers. Commented Aug 28, 2023 at 21:45
  • -bash: pyenv: command not found. Downvote.
    – John Smith
    Commented Apr 13 at 5:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.