265

I'm working on a documentation (personal) for nested matplotlib (MPL) library, which differs from MPL own provided, by interested submodule packages. I'm writing Python script which I hope will automate document generation from future MPL releases.

I selected interested submodules/packages and want to list their main classes from which I'll generate list and process it with pydoc.

The problem is that I can't find a way to instruct Python to load a submodule from a string. Here is an example of what I tried:

import matplotlib.text as text
x = dir(text)
i = __import__('matplotlib.text')
y = dir(i)
j = __import__('matplotlib')
z = dir(j)

And here is a 3-way comparison of above lists through pprint:

enter image description here

I don't understand what's loaded in y object - it's base matplotlib plus something else, but it lacks information that I wanted and that is main classes from matplotlib.text package. It's the top blue coloured part on screenshot (x list).

3
  • Can you explain why you need to use __import__(str) rather than the standard import statemetn?
    – thesamet
    Commented Jan 3, 2012 at 21:53
  • It's because I'll process lists which items are MPL submodules and get their methods paths
    – theta
    Commented Jan 3, 2012 at 22:00
  • 21
    @thesamet - c'mon - there are endless ideas where you'd want this functionality. When you have a textual configuration of libraries, you can load them by name, which wouldn't quite work with the import statement. Here's one example of use: djangosnippets.org/snippets/3048 Commented May 12, 2014 at 4:55

7 Answers 7

360

The __import__ function can be a bit hard to understand.

If you change

i = __import__('matplotlib.text')

to

i = __import__('matplotlib.text', fromlist=[''])

then i will refer to matplotlib.text.

In Python 3.1 or later, you can use importlib:

import importlib

i = importlib.import_module("matplotlib.text")

Some notes

  • If you're trying to import something from a sub-folder e.g. ./feature/email.py, the code will look like importlib.import_module("feature.email")

  • Before Python 3.3 you could not import anything if there was no __init__.py in the folder with file you were trying to import (see caveats before deciding if you want to keep the file for backward compatibility e.g. with pytest).

10
  • 3
    importlib should be avaliable on pypi for < python2.7 Commented Aug 19, 2012 at 12:12
  • 64
    For anyone who's coming here from Google. It should be noted that if you're trying to import something from a sub-folder (for example, ./feature/email.py) the code will look like importlib.import_module("feature.email")
    – Seanny123
    Commented Dec 6, 2013 at 7:13
  • 15
    Finally, also remember that you can't import anything if there is no __init__.py in the folder with file you are trying to import.
    – Seanny123
    Commented Dec 6, 2013 at 7:31
  • 4
    @mzjn This is for import moduleName where moduleName is string. How about from moduleName import * ?
    – Nam G VU
    Commented May 30, 2017 at 6:47
  • 3
    Just found answer for my question here in case any one needs it stackoverflow.com/a/31306598/248616
    – Nam G VU
    Commented May 30, 2017 at 7:54
117

importlib.import_module is what you are looking for. It returns the imported module.

import importlib

# equiv. of your `import matplotlib.text as text`
text = importlib.import_module('matplotlib.text')

You can thereafter access anything in the module as text.myclass, text.myfunction, etc.

3
6

spent some time trying to import modules from a list, and this is the thread that got me most of the way there - but I didnt grasp the use of ___import____ -

so here's how to import a module from a string, and get the same behavior as just import. And try/except the error case, too. :)

  pipmodules = ['pycurl', 'ansible', 'bad_module_no_beer']
  for module in pipmodules:
      try:
          # because we want to import using a variable, do it this way
          module_obj = __import__(module)
          # create a global object containging our module
          globals()[module] = module_obj
      except ImportError:
          sys.stderr.write("ERROR: missing python module: " + module + "\n")
          sys.exit(1)

and yes, for python 2.7> you have other options - but for 2.6<, this works.

4

Apart from using the importlib one can also use exec method to import a module from a string variable.

Here I am showing an example of importing the combinations method from itertools package using the exec method:

MODULES = [
    ['itertools','combinations'],
]

for ITEM in MODULES:
    import_str = "from {0} import {1}".format(ITEM[0],', '.join(str(i) for i in ITEM[1:]))
    exec(import_str)

ar = list(combinations([1, 2, 3, 4], 2))
for elements in ar:
    print(elements)

Output:

(1, 2)
(1, 3)
(1, 4)
(2, 3)
(2, 4)
(3, 4)
2

You can also use exec built-in function that execute any string as a Python code.

In [1]: module = 'pandas'
   ...: function = 'DataFrame'
   ...: alias = 'DF'

In [2]: exec(f"from {module} import {function} as {alias}")

In [3]: DF
Out[3]: pandas.core.frame.DataFrame

For me this was the most readable way to solve my problem.

1

Module auto-install & import from list

Below script works fine with both submodules and pseudo submodules.

# PyPI imports
import pkg_resources, subprocess, sys

modules   = {'lxml.etree', 'pandas', 'screeninfo'}
required  = {m.split('.')[0] for m in modules}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing   = required - installed

if missing:
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', '--upgrade', 'pip'])
    subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])

for module in set.union(required, modules):
    globals()[module] = __import__(module)

Tests:

print(pandas.__version__)
print(lxml.etree.LXML_VERSION)
0

I developed these 3 useful functions:

def loadModule(moduleName):
    module = None
    try:
        import sys
        del sys.modules[moduleName]
    except BaseException as err:
        pass
    try:
        import importlib
        module = importlib.import_module(moduleName)
    except BaseException as err:
        serr = str(err)
        print("Error to load the module '" + moduleName + "': " + serr)
    return module

def reloadModule(moduleName):
    module = loadModule(moduleName)
    moduleName, modulePath = str(module).replace("' from '", "||").replace("<module '", '').replace("'>", '').split("||")
    if (modulePath.endswith(".pyc")):
        import os
        os.remove(modulePath)
        module = loadModule(moduleName)
    return module

def getInstance(moduleName, param1, param2, param3):
    module = reloadModule(moduleName)
    instance = eval("module." + moduleName + "(param1, param2, param3)")
    return instance

And everytime I want to reload a new instance I just have to call getInstance() like this:

myInstance = getInstance("MyModule", myParam1, myParam2, myParam3)

Finally I can call all the functions inside the new Instance:

myInstance.aFunction()

The only specificity here is to customize the params list (param1, param2, param3) of your instance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.