271

When would the -e, or --editable option be useful with pip install?

For some projects the last line in requirements.txt is -e .. What does it do exactly?

6

7 Answers 7

222

As the man page says it:

-e,--editable <path/url>
     Install a project in editable mode (i.e.  setuptools "develop mode") from a local project path or a VCS url.

So you would use this when trying to install a package locally, most often in the case when you are developing it on your system. It will just link the package to the original location, basically meaning any changes to the original package would reflect directly in your environment.

Some nuggets around the same here and here.

An example run can be:

pip install -e .

or

pip install -e ~/ultimate-utils/ultimate-utils-proj-src/

note the second is the full path to where the setup.py would be at.

14
  • 88
    It is still hard to understand. Of course I read the --help page. But it didnt help. Lets say I just cloned a repo called 'abc'. And I install requirements.txt which contains -e .. Will it make some package from setup.py editable in site-packages? Sorry, need an example maybe.
    – raitisd
    Jan 28, 2016 at 14:57
  • 18
    @Raituha Hmm I guess the documentation could be a bit more verbose on this Jan 28, 2016 at 14:58
  • 43
    @variable: If you install your local project with -e option (pip install -e mypackage) and use it in your environment (e.g. within your other project like from mypackage import custom_function) then, when you make any change to your custom_function, you will able to use this updated version without re-installing it again (with pip install or python setup.py), which would happen in case of omitting -e flag.
    – Nerxis
    Apr 17, 2020 at 15:40
  • 7
    @raitisd: When you run pip install -r requirements.txt, it will install all required packages and then (if there is -e .) it should install current package in develop mode (e.g. you are in mypackage folder and it's equivalent of running pip install -e ., so any change in mypackage is directly reflected in your environment). No other packages are touched by this.
    – Nerxis
    Apr 17, 2020 at 15:47
  • 1
    @saper0 Yes, it will. And that's exactly why is this option there (you are developing something and you want to test it somewhere). Just try it by yourself.
    – Nerxis
    Feb 16, 2022 at 14:25
48

Concrete example of using --editable in development

If you play with this test package as in:

cd ~
git clone https://github.com/cirosantilli/vcdvcd
cd vcdvcd
git checkout 5dd4205c37ed0244ecaf443d8106fadb2f9cfbb8
python -m pip install --editable . --user

it outputs:

Obtaining file:///home/ciro/bak/git/vcdvcd
Installing collected packages: vcdvcd
  Attempting uninstall: vcdvcd
    Found existing installation: vcdvcd 1.0.6
    Can't uninstall 'vcdvcd'. No files were found to uninstall.
  Running setup.py develop for vcdvcd
Successfully installed vcdvcd-1.0.6

The Can't uninstall 'vcdvcd' is normal: it tried to uninstall any existing vcdvcd to then replace them with the "symlink-like mechanism" that is produced in the following steps, but failed because there were no previous installations.

Then it generates a file:

~/.local/lib/python3.8/site-packages/vcdvcd.egg-link

which contains:

/home/ciro/vcdvcd
.

and acts as a "symlink" to the Python interpreter.

So now, if I make any changes to the git source code under /home/ciro/vcdvcd, it reflects automatically on importers who can from any directory do:

python -c 'import vcdvcd'

Note however that at my pip version at least, binary files installed with --editable, such as the vcdcat script provided by that package via scripts= on setup.py, do not get symlinked, just copied to:

~/.local/bin/vcdcat

just like for regular installs, and therefore updates to the git repository won't directly affect them.

By comparison, a regular non --editable install from the git source:

python -m pip uninstall vcdvcd
python -m pip install --user .

produces a copy of the installed files under:

~/.local/lib/python3.8/site-packages/vcdvcd

Uninstall of an editable package as done above requires a new enough pip as mentioned at: How to uninstall editable packages with pip (installed with -e)

Tested in Python 3.8, pip 20.0.2, Ubuntu 20.04.

Recommendation: develop directly in-tree whenever possible

The editable setup is useful when you are testing your patch to a package through another project.

If however you can fully test your change in-tree, just do that instead of generating an editable install which is more complex.

E.g., the vcdvcd package above is setup in a way that you can just cd into the source and do ./vcdcat without pip installing the package itself (in general, you might need to install dependencies from requirements.txt though), and the import vcdvcd that that executable does (or possibly your own custom test) just finds the package correctly in the same directory it lives in.

4
  • 3
    'Then it generates a file: ~/.local/lib/python3.8/site-packages/vcdvcd.egg-link [...] and acts as a "symlink" to the Python interpreter.' --- This is what i was looking for, thank you!
    – sunyata
    Feb 21, 2021 at 13:18
  • can you give a concrete example of how to call pip -e? e.g. do I need as an argument the path to the project where setup.py is or the setup.py itself? e.g. pip -e path/src/setup.py or pip -e path/src/? Aug 31, 2021 at 19:08
  • 1
    @CharlieParker have you seen the mentioned package: github.com/cirosantilli/vcdvcd/tree/… ? If you follow the git clone, that is as concrete as it gets. You will have cded into the setup.py directory. Aug 31, 2021 at 19:16
  • 1
    if you are looking for an example of how to call pip install -e without . you can do for example: pip install -e ~/ultimate-utils/ultimate-utils-proj-src/ where path/src is the path to the src of the project where setup.py is at. Aug 31, 2021 at 21:26
13

From Working in "development" mode:

Although not required, it’s common to locally install your project in “editable” or “develop” mode while you’re working on it. This allows your project to be both installed and editable in project form.

Assuming you’re in the root of your project directory, then run:

pip install -e .

Although somewhat cryptic, -e is short for --editable, and . refers to the current working directory, so together, it means to install the current directory (i.e. your project) in editable mode.

Some additional insights into the internals of setuptools and distutils from “Development Mode”:

Under normal circumstances, the distutils assume that you are going to build a distribution of your project, not use it in its “raw” or “unbuilt” form. If you were to use the distutils that way, you would have to rebuild and reinstall your project every time you made a change to it during development.

Another problem that sometimes comes up with the distutils is that you may need to do development on two related projects at the same time. You may need to put both projects’ packages in the same directory to run them, but need to keep them separate for revision control purposes. How can you do this?

Setuptools allows you to deploy your projects for use in a common directory or staging area, but without copying any files. Thus, you can edit each project’s code in its checkout directory, and only need to run build commands when you change a project’s C extensions or similarly compiled files. You can even deploy a project into another project’s checkout directory, if that’s your preferred way of working (as opposed to using a common independent staging area or the site-packages directory).

To do this, use the setup.py develop command. It works very similarly to setup.py install, except that it doesn’t actually install anything. Instead, it creates a special .egg-link file in the deployment directory, that links to your project’s source code. And, if your deployment directory is Python’s site-packages directory, it will also update the easy-install.pth file to include your project’s source code, thereby making it available on sys.path for all programs using that Python installation.

4
  • Are edit mode, editable mode, develop mode, and development mode all the same thing? Dec 30, 2020 at 23:17
  • 1
    @KyleDelaney I believe so, but before giving you a false answer I would suggest you open a new question on SO. I would also be interested in the answer so feel free to post the link here as well. Jan 11, 2021 at 18:20
  • can you give a concrete example of how to call pip -e? e.g. do I need as an argument the path to the project where setup.py is or the setup.py itself? e.g. pip -e path/src/setup.py or pip -e path/src/? Aug 31, 2021 at 19:08
  • if you are looking for an example of how to call pip install -e without . you can do for example: pip install -e ~/ultimate-utils/ultimate-utils-proj-src/ where path/src is the path to the src of the project where setup.py is at. Aug 31, 2021 at 21:26
9

It is important to note that pip uninstall can not uninstall a module that has been installed with pip install -e. So if you go down this route, be prepared for things to get very messy if you ever need to uninstall. A partial solution is to (1) reinstall, keeping a record of files created, as in sudo python3 -m setup.py install --record installed_files.txt, and then (2) manually delete all the files listed, as in e.g. sudo rm -r /usr/local/lib/python3.7/dist-packages/tdc7201-0.1a2-py3.7.egg/ (for release 0.1a2 of module tdc7201). This does not 100% clean everything up however; even after you've done it, importing the (removed!) local library may succeed, and attempting to install the same version from a remote server may fail to do anything (because it thinks your (deleted!) local version is already up to date).

2
  • 1
    Interesting point, so essentially install with --editable is potentially irreversible? Does that mean we need to simply abandon the environment then? May 13, 2020 at 19:10
  • 30
    It should be noted that this is no longer true as of recent versions of pip. See the following stack overflow answer comment: stackoverflow.com/questions/17346619/…
    – root
    Jun 27, 2020 at 2:26
2

Still none of the duplicates explain in a proper way what pip install -e . or -e . in requirements.txt means.

I understand pip install -e package installs the package in editable mode, placing a link in the venv site packages.
But it is not so clear when . is specified instead of package.

Where is the package installed then, where is the link placed, if at all?
What is the difference between pip install -r reqs.txt with the reqs in the reqs.txt and pip install -e . with the reqs in the setup.cfg?

The comment by Wör Du Schnaffzig brings up the need for a clearer explanation of what happens when you use pip install -e . or specify -e . in a requirements.txt file, as opposed to specifying a package name.


When pip install -e . is executed, the . represents the current directory you are in. That is typically the root directory of the Python package you are developing.

What pip does in this case is that it installs the package in a way that links back to this directory, rather than copying the files to the site-packages directory. That means that any changes you make to the source code will be reflected when you import the package in Python, without having to reinstall the package.

  • pip locates the setup.py (legacy) or pyproject.toml file in the current directory (the .).
  • Instead of copying the contents to site-packages, pip creates a file in site-packages that points to the current project directory (this is the "link" that is mentioned).
  • The link is usually an .egg-link file (mentioned, for instance, in pypa/pip issue 10476)which contains the path to the project directory and a simple . indicating that this directory is the one to be added to sys.path.

When -e . is included in a requirements.txt file and pip install -r requirements.txt is run, it does the same thing as pip install -e . but as part of processing the requirements file.


To directly address Wör Du Schnaffzig's questions:

  • When . is specified, the "package" is your project in the current directory. It is "installed" in such a way that Python will use the source code in the current directory instead of the code in site-packages.
  • The link (in the form of an .egg-link file) is placed in the site-packages directory of the current Python environment (whether that is a virtual environment or the global Python environment).
  • The difference between pip install -r reqs.txt with requirements specified inside reqs.txt and pip install -e . with requirements in setup.cfg is that:
    • the former (pip install -r reqs.txt) installs the packages listed in reqs.txt normally (copying them to site-packages),
    • while the latter (pip install -e .) installs the current directory as an editable package and processes dependencies as specified in setup.cfg or setup.py.
+-----------------+      +------------------+
| Current Dir (.) | ---> | pip install -e . |
+-----------------+      +------------------+
             |
             | (Editable Mode Link)
             v
+----------------------------+      +-----------------------+
| .egg-link in site-packages | ---> | Source Code Directory |
+----------------------------+      +-----------------------+
             |
             | (Immediate Reflection of Changes)
             v
+--------------------+
| Python Environment |
+--------------------+

In practice, using -e . is like telling Python, "Please always refer back to this specific directory for the code of this package, even if I'm running Python code from somewhere else."

The site-packages directory will contain a reference to the package's source code directory, not the package code itself. That allows you to work on your codebase and have those changes immediately available everywhere that Python environment is used without reinstallation.

See also "setuptools / Development Mode (a.k.a. “Editable Installs”)"

1

As suggested in previous answers, there is no symlinks that are getting created. How does '-e' option work? -> It just updates the file "PYTHONDIR/site-packages/easy-install.pth" with the project path specified in the 'command pip install -e'. So each time python search for a package it will check this directory as well => any changes to the files in this directory is instantly reflected.

1
  • I think this applies only to projects using a setup.py. With pyproject.toml based projects, the entry to the easy-install.pth file is not added unless using the deprecated compat mode with pip. This has the implication that some IDE tools will not find pyproject.toml based editable installed packages without some extra help (like: adding a .pth file entry).
    – Niko Fohr
    Dec 8, 2023 at 7:31
0

The chief reason this is useful is because it frees your development from having to bother with all the repeated pip installs from local sources.

Imagine you have to fix a bug that requires an API change to three packages. You edit one of them, pip install it locally, test the other one on these new changes to the first, edit the second and likewise install it locally and go to edit the third package find that it has broke between the two edits and you need to go back and edit the first again, pip install etc etc. That’s a lot of development friction to have to bother with all these intermediate pip install steps.

What is more desirable in this type of multi package dependency package development is to instead install them all in editable mode. Then you can freely edit and make changes and test without ever needing to do the intermediate pip installs to get the packages working off your latest draft. You do the upfront editable install from source, get to hacking and testing and when everything is proven you can commit the changes to all packages and ship a parallel release for all packages.

Also another reason is it helps with code navigation in IDEs where you always want to navigate to dependency packages in a repo, especially a monorepo, where multiple packages live. You can enumerate in configurations many includes folders for the IDE to know of the many locations where a package can live or the interpreter that is selected in the IDE can have editable installs set to the clone of your repo and likewise be able to navigate to folders especially if those package folders do aliasing in their setup.py logic.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.