504

I need to install a package from PyPI straight within my script.

Is there maybe some module or distutils (distribute, pip, etc.) feature which allows me to just execute something like pypi.install('requests') and requests will be installed into my virtualenv?

11
  • 4
    Why don't you define the related module as dependency in the setup.py of your own package?
    – user2665694
    Sep 8, 2012 at 17:34
  • 8
    you know ... the way its actually supposed to be done ... but you could always os.system("pip install blah") but you may need sudo access ... better to just make it a dependency in your setup.py Sep 8, 2012 at 17:35
  • 2
    Would you consider changing the accepted answer here? Importing pip is never a good idea - the mere fact that all of its contents are in _internal starting from version 10... Dec 6, 2019 at 21:53
  • 2
    @chuwy stackoverflow.com/a/50255019/918959 <= this one. pip._internal is not designed to be importable, it can do absolutely random things when imported in another program. Dec 11, 2019 at 10:26
  • 3
    @AnttiHaapala okay I changed it. I personally don't like both solutions, but I'm far away from Python these days, so I trust your opinion.
    – chuwy
    Dec 13, 2019 at 17:12

13 Answers 13

591
+50

The officially recommended way to install packages from a script is by calling pip's command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal precisely in order to make it clear to users that programmatic use of pip is not allowed.

Use sys.executable to ensure that you will call the same pip associated with the current runtime.

import subprocess
import sys

def install(package):
    subprocess.check_call([sys.executable, "-m", "pip", "install", package])
8
  • 4
    One issue with this is that, for novice users on Windows, python and pip are not always on their PATH, and so a .py file that could be double-clicked would be quite convenient, whereas a "pip install xxx" comment can be quite tricky.
    – jdpipe
    Apr 17, 2020 at 1:27
  • 16
    CalledProcessError: Command '['C:\\ProgramData\\Anaconda3\\pythonw.exe', '-m', 'pip', 'install', 'googleapiclient']' returned non-zero exit status 1.
    – parvij
    May 13, 2020 at 18:30
  • 2
    I'm trying to use this approach, but my python is embedded/started from another executable, so "sys.executable" doesn't return the right path. Is there an alternative that would work for python that's started by some other process?
    – user441669
    Dec 26, 2020 at 23:56
  • strange thing I need to rerun the script the module was installed from so that it will be recognized
    – BND
    Jun 24, 2022 at 7:53
  • 11
    note, since 3.5 its now recommended to use subprocess.run(..., check=True)
    – Brian
    Jul 22, 2022 at 12:18
438

You can also use something like:

import pip

def install(package):
    if hasattr(pip, 'main'):
        pip.main(['install', package])
    else:
        pip._internal.main(['install', package])

# Example
if __name__ == '__main__':
    install('argh')
24
  • 3
    @nbro you pass options to pip.main() as you would on the command line (but with each option as a separate entry in the list, instead of a single string). And to specify which version of the package you want, do the same as on the command line.. ex: pip.main(['install', 'pip==7.1.0'])
    – Kaos
    Jul 20, 2015 at 10:18
  • 3
    See also stackoverflow.com/questions/6457794/…, which shows how to handle the case where an install fails.
    – Myer
    Jul 20, 2015 at 11:21
  • 31
    from pip._internal import main as pipmain then you can use pipmain() just like the deprecated pip.main() see stackoverflow.com/questions/43398961/…
    – 3pitt
    May 30, 2018 at 15:48
  • 19
    its depreceated now.
    – Morse
    Jul 8, 2018 at 1:12
  • 14
    It's deprecated for a reason & not recommended anymore. see pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program
    – nbk
    Jul 28, 2019 at 16:56
90

If you want to use pip to install required package and import it after installation, you can use this code:

def install_and_import(package):
    import importlib
    try:
        importlib.import_module(package)
    except ImportError:
        import pip
        pip.main(['install', package])
    finally:
        globals()[package] = importlib.import_module(package)


install_and_import('transliterate')

If you installed a package as a user you can encounter the problem that you cannot just import the package. See How to refresh sys.path? for additional information.

6
  • 4
    Any idea how to do that on Python 3? imp.reload(site) gets me RuntimeError: dictionary changed size during iteration
    – kgadek
    Aug 3, 2015 at 16:35
  • where does this install the package, after i did this, i was not able to do pip uninstall <package_name>. I can still uninstall it using pip.main but just wanted to know where does it install the package? Apr 26, 2016 at 12:20
  • Was curious. would this work properly if i do: pip install requests[security] ? I wasnt sure if it would properly define the globals correctly. May 8, 2017 at 16:47
  • 7
    Outdated. pip.main no longer works. pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program
    – wim
    Dec 5, 2019 at 16:10
  • 1
    Does importing within a function really import into the main namespace, or just the namespace of that install_and_import function?
    – sh37211
    Mar 31, 2021 at 0:26
47

This should work:

import subprocess

def install(name):
    subprocess.call([sys.executable, '-m', 'pip', 'install', name])
10
  • 2
    Yes it's definitely should work. But I thought there is more elegant way;) I'll be waiting a little bit, may be there is.
    – chuwy
    Sep 8, 2012 at 18:03
  • 16
    It depends on the right version of pip being first on the path. If the user is running an alternate python installation, pip will install into the first one instead of the current one. The import approach above will install in the right place. I upvoted anyway to counter the down vote.
    – GaryBishop
    Jan 17, 2014 at 11:57
  • 7
    Depending on how the script is running you wont call the right pip.
    – Natim
    Jan 13, 2016 at 13:13
  • 1
    There is a lot of situations when this approach will not work. e.g. when you use several versions of python on one machine and you interpret current script with not default python.
    – running.t
    Jan 10, 2017 at 14:38
  • 13
    Calling [sys.executable, '-m', 'pip', 'install', name] is making sure to get the "right" pip here.
    – wim
    Dec 5, 2019 at 16:11
40

I added some exception handling to Aaron's answer.

import subprocess
import sys

try:
    import pandas as pd
except ImportError:
    subprocess.check_call([sys.executable, "-m", "pip", "install", 'pandas'])
finally:
    import pandas as pd
5
  • 2
    nice implementation of subprocess and pip, better than most solutions here Sep 23, 2019 at 11:30
  • 4
    You're not checking the retunr value of subprocess.call so the code might fail. Dec 7, 2019 at 13:42
  • 1
    why subprocess.check_call and not subprocess.call?
    – Coddy
    Mar 12, 2021 at 4:50
  • Ok, running "import pandas as pd" brings no problem, but... logically isn't it ugly? Oct 4, 2021 at 13:14
  • 1
    Nice, but I would do the last import after the installation of the except-clause. Apr 28, 2023 at 9:57
29

For installing multiple packages, I am using a setup.py file with the following code:

import sys
import subprocess
import pkg_resources

required  = {'numpy', 'pandas', '<etc>'} 
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    # implement pip as a subprocess:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])
2
13

Use:

import os
os.system('pip install requests')

I tried the above for a temporary solution instead of changing a Docker file.

1
  • this is so easy and simple to understand for beginners as compared to all the other answers. Thank you very much. Nov 3, 2021 at 9:17
8

If you want a more efficient answer that expands on subprocess.check_call. You can first check if the requirement has already been met using pkg_resources.

This works for different requirement specifiers which is nice. E.g., >=, ==:

import sys
import subprocess
import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict

def should_install_requirement(requirement):
    should_install = False
    try:
        pkg_resources.require(requirement)
    except (DistributionNotFound, VersionConflict):
        should_install = True
    return should_install


def install_packages(requirement_list):
    try:
        requirements = [
            requirement
            for requirement in requirement_list
            if should_install_requirement(requirement)
        ]
        if len(requirements) > 0:
            subprocess.check_call([sys.executable, "-m", "pip", "install", *requirements])
        else:
            print("Requirements already satisfied.")

    except Exception as e:
        print(e)

Example usage:

requirement_list = ['requests', 'httpx==0.18.2']
install_packages(requirement_list)

Demo

Relevant information: Stack Overflow question: 58612272

2
  • Amazing suggestion! As a little improvement, here we go: ``` if requirements := [requirement for requirement in requirement_list if should_install_requirement(requirement)]: subprocess.check_call( [sys.executable, "-m", "pip", "install", *requirements]) else: print("Requirements already satisfied.") ``` Jul 26, 2022 at 16:12
  • The animation is unreadable (though it does work if opened in a separate tab or window). Feb 23, 2023 at 5:39
7

You define the dependent module inside the setup.py of your own package with the "install_requires" option.

If your package needs to have some console script generated then you can use the "console_scripts" entry point in order to generate a wrapper script that will be placed within the 'bin' folder (e.g. of your virtualenv environment).

3
  • 1
    This is the correct answer and the only sensible way to manage a Python projects' dependencies. It will work with virtualenv, Fabric, buildout, you name it. The method described by @xiaomao, even though answering exactly what the OP asked, is pure madness.
    – Lukas Graf
    Sep 8, 2012 at 20:48
  • 9
    while this is proper advice, it doesn't answer the question asked Jan 20, 2017 at 16:29
  • 4
    While packaging is a topic, there are a lot of other use cases, for example a deployment script written in python.
    – hoefling
    Feb 2, 2017 at 18:58
1

To conditionally install multiple packages with exact version, I've been using this pattern basing on @Tanmay Shrivastava's answer:

import sys
from subprocess import run, PIPE, STDOUT
import pkg_resources

def run_cmd(cmd):
    ps = run(cmd, stdout=PIPE, stderr=STDOUT, shell=True, text=True)
    print(ps.stdout)


# packages to be conditionally installed with exact version
required = {"click==8.0.1", "semver==3.0.0.dev2"}
installed = {f"{pkg.key}=={pkg.version}" for pkg in pkg_resources.working_set}
missing = required - installed

if missing:
    run_cmd(f'pip install --ignore-installed {" ".join([*missing])}')
1
    import pip

try:
    import imaplib
    import email
    import pandas as pd
    # for hiding password
    from pathlib import Path
    from dotenv import load_dotenv
    import os
    import requests
    # 
    from collections import defaultdict
    from itertools import permutations,combinations
except Exception as e:
    print(e)
    e = str(e).split(' ')[-1].replace("'","")
    pip.main(['install', e])
1

I didn't like all the suggested options and I wrote my library for this.

Install it:

pip install instld

And use a context manager:

import installed

with installed('some_package'):
  import some_module

To learn more about the library, follow the link I gave above. It allows you to use two different versions of the same library in one program, or, for example, libraries that are incompatible with each other.

0

Try the below. So far, it was the best that worked for me.

Install the 4 ones first and then mention the new ones in the "REQUIRED" list:

import pkg_resources
import subprocess
import sys
import os

REQUIRED = {
  'spacy', 'scikit-learn', 'numpy', 'pandas', 'torch',
  'pyfunctional', 'textblob', 'seaborn', 'matplotlib'
}

installed = {pkg.key for pkg in pkg_resources.working_set}
missing = REQUIRED - installed

if missing:
    python = sys.executable
    subprocess.check_call([python, '-m', 'pip', 'install', *missing], stdout=subprocess.DEVNULL)
1
  • 3
    This is just a copy of Tanmay Shrivastava's answer Aug 8, 2021 at 15:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.