150

I have a git repository with many folders, one of them being a python module installable with pip, like this:

repo.git/
repo.git/folder1/
repo.git/folder2/
repo.git/mymodule/
repo.git/mymodule/__init__.py
repo.git/mymodule/setup.py
repo.git/mymodule/...

Right now I have to do the following to install:

git clone http://server/repo.git
cd repo
pip install mymodule
cd ..
rm -rf repo

Is it possible to install the module directly with pip without explicitly cloning ?

I tried:

pip install git+https://server/repo.git/mymodule/
pip install git+https://server/repo.git:mymodule/

But I get:

IOError: [Errno 2] No such file or directory: '/tmp/pip-88tlLm-build/setup.py'
3
  • 1
    This question seems to already be answered: stackoverflow.com/questions/10847764/pip-install-from-git-repo Nov 26, 2012 at 20:47
  • 4
    The question you linked was about missing files because there's no MANIFEST.in. My question is about how to install a subdirectory with pip, no files are missing since I can install the package by cloning the whole repo. Nov 27, 2012 at 7:59
  • 1
    @synthesizerpatel The question you linked to asks about different thing but the title was misleading so I corrected it. You can consider removing your (I guess) downvote. Nov 27, 2012 at 21:29

2 Answers 2

191

There is a pull request regarding this feature, and it seems to have been merged to develop branch a month ago. The syntax is the following:

pip install -e "git+https://git.repo/some_repo.git#egg=$NAME_OF_PACKAGE&subdirectory=$SUBDIR_IN_REPO" # install a python package from a repo subdirectory

We probably have to wait for a while until it gets merged to master and is distributed.

UPDATE: This is now available and documented at https://pip.pypa.io/en/stable/cli/pip_install/#vcs-support as follows:

For projects where setup.py is not in the root of project, "subdirectory" component is used. Value of "subdirectory" component should be a path starting from root of the project to where setup.py is located.

So if your repository layout is:

- pkg_dir/
  - setup.py  # setup.py for package ``pkg``
  - some_module.py
- other_dir/
  - some_file
- some_other_file

You'll need to use

pip install -e "vcs+protocol://repo_url/#egg=pkg&subdirectory=pkg_dir"

Note: Make sure to surround it with quotes otherwise subdirectory won't work.

10
  • 4
    I just checked and it is part of pip v1.5, released Jan 1 2014: github.com/pypa/pip/blob/develop/CHANGES.txt
    – tomka
    Feb 10, 2014 at 14:10
  • 26
    If you are using ubuntu make sure to use single quotation marks before the actual git location, otherwise the &subdirectory is ignored! For example: pip install -e 'git+git.repo/…'
    – Lin
    Nov 9, 2014 at 12:21
  • 12
    Just a heads-up: this does not seem to work (at least on pip v1.5.6, not sure about 6+ versions) for packages whose entry points lie within the subdirectory. pip install -e 'git+https://…/repo.git@branch#egg=package&subdirectory=package' results in the package appearing to install, and a package.egg-link file getting put in site-packages — but it points to the root directory of src/repo, not to src/repo/package, as I would expect. This means that py_modules can’t be imported, and entry_points don’t actually work: they both raise an ImportError: No module named package.
    – Mark G.
    Dec 7, 2015 at 17:57
  • 2
    -e seems necessary, otherwise pip freeze doesn't show the whole path, only the egg. This happened on pip-1.5.4
    – zyxue
    Mar 26, 2016 at 15:29
  • 2
    tip for others: you HAVE to use git+https, git+git will not work, you'll get fatal: remote error: Repository not found. presumably because it doesn't parse the "query" string correctly. I've just logged: github.com/pypa/pip/issues/4898
    – amohr
    Nov 30, 2017 at 22:45
48

It's been already stated in one of the comments under the correct answer, but just to highlight this issue: when executing this from Linux command line, you must escape the &-character since ampersand is telling the command line to run a command in background:

git+https://git.repo/some_repo.git#egg=version_subpkg\&subdirectory=repo

Notice the backslash before the ampersand. The escaping behaviour might depend on the Linux distro; I'm not an expert.
If you ignore this, you might run into a cryptic error like the following:

bash: (...) command not found
5
  • 3
    Would enclosing the url in single quotes help avoid the need for escaping the ampersand? I know that is helpful when working with URLs in cURL.
    – Josh Peak
    Oct 22, 2018 at 21:43
  • 1
    Also worth noting: When executing this from Windows command line, you must place the URL in double quotes, or you'll get an error "'subdirectory' is not recognized as an internal or external command". E.g., use pip install -e "git+myurl/proj#egg=subpkg&subdirectory=subdir"
    – user447688
    Dec 4, 2018 at 16:02
  • In general, when working with shell scripts, use double quotes around arguments to be on the safe side. for more best practices see: shellcheck.net or github.com/koalaman/shellcheck Jan 17, 2019 at 15:38
  • 2
    The escaping is necessary in OSX as well (as it is a sort of Linux distro)
    – ygesher
    Mar 3, 2019 at 16:53
  • 13
    For branch-based install use: pip install git+ssh://[email protected]/org_or_username/repo.git@branch#subdirectory=path/to/dubdir
    – RDK
    Sep 6, 2019 at 6:41

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.