549

PEP 8 states:

Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.

However if the class/method/function that I am importing is only used in rare cases, surely it is more efficient to do the import when it is needed?

Isn't this:

class SomeClass(object):

    def not_often_called(self)
        from datetime import datetime
        self.datetime = datetime.now()

more efficient than this?

from datetime import datetime

class SomeClass(object):

    def not_often_called(self)
        self.datetime = datetime.now()

22 Answers 22

389

Module importing is quite fast, but not instant. This means that:

  • Putting the imports at the top of the module is fine, because it's a trivial cost that's only paid once.
  • Putting the imports within a function will cause calls to that function to take longer.

So if you care about efficiency, put the imports at the top. Only move them into a function if your profiling shows that would help (you did profile to see where best to improve performance, right??)


The best reasons I've seen to perform lazy imports are:

  • Optional library support. If your code has multiple paths that use different libraries, don't break if an optional library is not installed.
  • In the __init__.py of a plugin, which might be imported but not actually used. Examples are Bazaar plugins, which use bzrlib's lazy-loading framework.
5
  • 28
    This answer focuses on the import statement cost, but not the cost of what is being imported. Take into consideration the library being imported. Some libraries may run expensive operations (in runtime or memory costs) on import.
    – Dennis
    Commented Nov 7, 2018 at 13:24
  • @Dennis, that's because the initial cost of import is constant no matter when you do it. The actual loading process you are referring to does not happen a second time. The answer is referring to the check that has to be done in sys.modules when an import statement is encountered. Commented Jan 9, 2021 at 7:41
  • 20
    I disagree. Sometimes you care about this cost you "pay once", as you pay it every time you start your program, and before your code starts doing anything useful, and some common modules take a noticeable time to import. Take numpy, pyplot, cv2, pandas, each of those modules take a time between 0.2 and 1 second to import on my PC. This makes the startup quite sluggish. On the other hand, an import inside a function will be unnoticeable unless you put it into some critical code that you should not code in python. Reimporting the same module 1000000 times takes about 0.1s.
    – bct
    Commented Apr 27, 2021 at 8:54
  • 2
    another practical reason to do something other than putting all imports at the top is when the import is qualified by input to the program. I had a debugging case where is it was desirable to separate the main program from its usual directory structure. This involved making the program reactive to whether or not it was being debugged at start-up which required processing sys.path before doing the import. However, on reflection, if the conditional information had been in an environment variable, the imports could have still been done at the global scope. Hmm... I may have to go change some code
    – cycollins
    Commented Jun 15, 2021 at 4:44
  • This answer is misleading, and I'd say the very first phrase "Module importing is quite fast" is really incorrect. When you import numpy or sympy, this takes quite some time and CPU. (Fortunately, it is done only once and not each time the import statement is seen.)
    – Max
    Commented Mar 22, 2022 at 4:51
109

Putting the import statement inside of a function can prevent circular dependencies. For example, if you have 2 modules, X.py and Y.py, and they both need to import each other, this will cause a circular dependency when you import one of the modules causing an infinite loop. If you move the import statement in one of the modules then it won't try to import the other module till the function is called, and that module will already be imported, so no infinite loop. Read here for more - effbot.org/zone/import-confusion.htm

4
  • 3
    Yes but one can get into dependency hell.
    – eigenein
    Commented Nov 17, 2014 at 16:13
  • 67
    If two modules need to import each other, something is seriously wrong with the code.
    – Anna
    Commented Aug 21, 2018 at 11:41
  • 5
    Object oriented programming often leads me to circular dependencies. A vital object class may be imported into several modules. In order for this object to perform its own tasks, it may need to reach out to one or more of those modules. There are ways to avoid it, such as sending data to the object through function args, to allow it to access the other module. But there are times when doing this feels very counter-intuitive to OOP (the outside world shouldn't need to know how its accomplishing the task in that function).
    – Robert
    Commented Dec 2, 2019 at 15:55
  • 24
    When X needs Y and Y needs X, they are either two parts of the same idea (i.e. should be defined together) or there's a missing abstraction.
    – GLRoman
    Commented Jan 16, 2020 at 18:48
88

I have adopted the practice of putting all imports in the functions that use them, rather than at the top of the module.

The benefit I get is the ability to refactor more reliably. When I move a function from one module to another, I know that the function will continue to work with all of its legacy of testing intact. If I have my imports at the top of the module, when I move a function, I find that I end up spending a lot of time getting the new module's imports complete and minimal. A refactoring IDE might make this irrelevant.

There is a speed penalty as mentioned elsewhere. I have measured this in my application and found it to be insignificant for my purposes.

It is also nice to be able to see all module dependencies up front without resorting to search (e.g. grep). However, the reason I care about module dependencies is generally because I'm installing, refactoring, or moving an entire system comprising multiple files, not just a single module. In that case, I'm going to perform a global search anyway to make sure I have the system-level dependencies. So I have not found global imports to aid my understanding of a system in practice.

I usually put the import of sys inside the if __name__=='__main__' check and then pass arguments (like sys.argv[1:]) to a main() function. This allows me to use main in a context where sys has not been imported.

8
  • 4
    Many IDEs easy this the refactoring of code by optimizing and auto-importing necessary modules into your file for you. In the majority of the cases PyCharm and Eclipse have made the correct decisions for me. I would bet there is a way of getting the same behavior in emacs or vim. Commented Aug 31, 2011 at 21:13
  • 5
    An import inside of an if statement in the global namespace is still a global import. This will print the arguments (using Python 3): def main(): print(sys.argv); if True: import sys; main(); You would have to wrap if __name__=='__main__' in a function to create a new namespace.
    – Darcinon
    Commented Jan 5, 2016 at 19:38
  • 15
    This strikes me as an excellent reason to import within functions rather than in the global scope. I'm quite surprised nobody else has mentioned doing it for this same reason. Are there are any significant downsides, besides performance and verbosity?
    – algal
    Commented Jan 23, 2018 at 16:47
  • 2
    I found it extremely useful for refactoring to put imports close to where I use them. It is no longer necessary to scroll to the top and back so many tims. I use IDEs like pycharm or wing ide and also use their refactoring, but I do not always want to rely on them. Moving functions to another module becomes a lot easier with this alternative importing style, as a consequence I refactor much more.
    – mit
    Commented Nov 10, 2018 at 17:59
  • 2
    A major downside of this suggestion is you might not realize you've introduced a circular dependency. You'll then have a much bigger refactoring problem and even an API breakage problem to fix it.
    – Ehren
    Commented Nov 12, 2020 at 18:43
48

Most of the time this would be useful for clarity and sensible to do but it's not always the case. Below are a couple of examples of circumstances where module imports might live elsewhere.

Firstly, you could have a module with a unit test of the form:

if __name__ == '__main__':
    import foo
    aa = foo.xyz()         # initiate something for the test

Secondly, you might have a requirement to conditionally import some different module at runtime.

if [condition]:
    import foo as plugin_api
else:
    import bar as plugin_api
xx = plugin_api.Plugin()
[...]

There are probably other situations where you might place imports in other parts in the code.

19

The first variant is indeed more efficient than the second when the function is called either zero or one times. With the second and subsequent invocations, however, the "import every call" approach is actually less efficient. See this link for a lazy-loading technique that combines the best of both approaches by doing a "lazy import".

But there are reasons other than efficiency why you might prefer one over the other. One approach is makes it much more clear to someone reading the code as to the dependencies that this module has. They also have very different failure characteristics -- the first will fail at load time if there's no "datetime" module while the second won't fail until the method is called.

Added Note: In IronPython, imports can be quite a bit more expensive than in CPython because the code is basically being compiled as it's being imported.

0
19

Here's an updated summary of the answers to this and related questions.

  • PEP 8 recommends putting imports at the top.
  • It's often more convenient to get ImportErrors when you first run your program rather than when your program first calls your function.
  • Putting imports in the function scope can help avoid issues with circular imports.
  • Putting imports in the function scope helps keep maintain a clean module namespace, so that it does not appear among tab-completion suggestions.
  • Start-up time: imports in a function won't run until (if) that function is called. Might get significant with heavy-weight libraries.
  • Even though import statements are super fast on subsequent runs, they still incur a speed penalty which can be significant if the function is trivial but frequently in use.
  • Imports under the __name__ == "__main__" guard seem very reasonable.
  • Refactoring might be easier if the imports are located in the function where they're used (facilitates moving it to another module). It can also be argued that this is good for readability. However, most would argue the contrary, see the next item.
  • Imports at the top enhance readability, since you can see all your dependencies at a glance.
  • It seems unclear if dynamic (possibly conditional) imports favour one style over another.
9

Curt makes a good point: the second version is clearer and will fail at load time rather than later, and unexpectedly.

Normally I don't worry about the efficiency of loading modules, since it's (a) pretty fast, and (b) mostly only happens at startup.

If you have to load heavyweight modules at unexpected times, it probably makes more sense to load them dynamically with the __import__ function, and be sure to catch ImportError exceptions, and handle them in a reasonable manner.

8

I was surprised not to see actual cost numbers for the repeated load-checks posted already, although there are many good explanations of what to expect.

If you import at the top, you take the load hit no matter what. That's pretty small, but commonly in the milliseconds, not nanoseconds.

If you import within a function(s), then you only take the hit for loading if and when one of those functions is first called. As many have pointed out, if that doesn't happen at all, you save the load time. But if the function(s) get called a lot, you take a repeated though much smaller hit (for checking that it has been loaded; not for actually re-loading). On the other hand, as @aaronasterling pointed out you also save a little because importing within a function lets the function use slightly-faster local variable lookups to identify the name later (http://stackoverflow.com/questions/477096/python-import-coding-style/4789963#4789963).

Here are the results of a simple test that imports a few things from inside a function. The times reported (in Python 2.7.14 on a 2.3 GHz Intel Core i7) are shown below (the 2nd call taking more than later calls seems consistent, though I don't know why).

 0 foo:   14429.0924 µs
 1 foo:      63.8962 µs
 2 foo:      10.0136 µs
 3 foo:       7.1526 µs
 4 foo:       7.8678 µs
 0 bar:       9.0599 µs
 1 bar:       6.9141 µs
 2 bar:       7.1526 µs
 3 bar:       7.8678 µs
 4 bar:       7.1526 µs

The code:

from __future__ import print_function
from time import time

def foo():
    import collections
    import re
    import string
    import math
    import subprocess
    return

def bar():
    import collections
    import re
    import string
    import math
    import subprocess
    return

t0 = time()
for i in xrange(5):
    foo()
    t1 = time()
    print("    %2d foo: %12.4f \xC2\xB5s" % (i, (t1-t0)*1E6))
    t0 = t1
for i in xrange(5):
    bar()
    t1 = time()
    print("    %2d bar: %12.4f \xC2\xB5s" % (i, (t1-t0)*1E6))
    t0 = t1
1
  • 2
    The changes in runtime are likely due to CPU frequency scaling in response to the load. It's better to start speed tests with a second of busy work to get the CPU clock speed to scale up. Commented Jun 7, 2020 at 12:47
8

I wouldn't worry about the efficiency of loading the module up front too much. The memory taken up by the module won't be very big (assuming it's modular enough) and the startup cost will be negligible.

In most cases you want to load the modules at the top of the source file. For somebody reading your code, it makes it much easier to tell what function or object came from what module.

One good reason to import a module elsewhere in the code is if it's used in a debugging statement.

For example:

do_something_with_x(x)

I could debug this with:

from pprint import pprint
pprint(x)
do_something_with_x(x)

Of course, the other reason to import modules elsewhere in the code is if you need to dynamically import them. This is because you pretty much don't have any choice.

I wouldn't worry about the efficiency of loading the module up front too much. The memory taken up by the module won't be very big (assuming it's modular enough) and the startup cost will be negligible.

1
  • We're talking about tens of milliseconds of startup cost per module (on my machine). That's not always negligible, e.g. if it affects the responsiveness of a web application to a user click. Commented Mar 26, 2016 at 4:01
6

It's a tradeoff, that only the programmer can decide to make.

Case 1 saves some memory and startup time by not importing the datetime module (and doing whatever initialization it might require) until needed. Note that doing the import 'only when called' also means doing it 'every time when called', so each call after the first one is still incurring the additional overhead of doing the import.

Case 2 save some execution time and latency by importing datetime beforehand so that not_often_called() will return more quickly when it is called, and also by not incurring the overhead of an import on every call.

Besides efficiency, it's easier to see module dependencies up front if the import statements are ... up front. Hiding them down in the code can make it more difficult to easily find what modules something depends on.

Personally I generally follow the PEP except for things like unit tests and such that I don't want always loaded because I know they aren't going to be used except for test code.

1
  • 2
    -1. The main overhead of importing only occurs the first time. The cost of looking up the module in sys.modules can easily be offset by the savings in only having to lookup a local name instead of a global name. Commented Jan 25, 2011 at 4:31
6

Here's an example where all the imports are at the very top (this is the only time I've needed to do this). I want to be able to terminate a subprocess on both Un*x and Windows.

import os
# ...
try:
    kill = os.kill  # will raise AttributeError on Windows
    from signal import SIGTERM
    def terminate(process):
        kill(process.pid, SIGTERM)
except (AttributeError, ImportError):
    try:
        from win32api import TerminateProcess  # use win32api if available
        def terminate(process):
            TerminateProcess(int(process._handle), -1)
    except ImportError:
        def terminate(process):
            raise NotImplementedError  # define a dummy function

(On review: what John Millikin said.)

6

This is like many other optimizations - you sacrifice some readability for speed. As John mentioned, if you've done your profiling homework and found this to be a significantly useful enough change and you need the extra speed, then go for it. It'd probably be good to put a note up with all the other imports:

from foo import bar
from baz import qux
# Note: datetime is imported in SomeClass below
1
  • That seems like a code smell to me. One of the points of all this is decluttering: adding comments like the above seem to go against this spirit of decluttering. Putting the import into SomeClass will make it clear enough where datetime is imported.
    – eric
    Commented Oct 7, 2022 at 14:51
5

Module initialization only occurs once - on the first import. If the module in question is from the standard library, then you will likely import it from other modules in your program as well. For a module as prevalent as datetime, it is also likely a dependency for a slew of other standard libraries. The import statement would cost very little then since the module intialization would have happened already. All it is doing at this point is binding the existing module object to the local scope.

Couple that information with the argument for readability and I would say that it is best to have the import statement at module scope.

5

Just to complete Moe's answer and the original question:

When we have to deal with circular dependences we can do some "tricks". Assuming we're working with modules a.py and b.py that contain x() and b y(), respectively. Then:

  1. We can move one of the from imports at the bottom of the module.
  2. We can move one of the from imports inside the function or method that is actually requiring the import (this isn't always possible, as you may use it from several places).
  3. We can change one of the two from imports to be an import that looks like: import a

So, to conclude. If you aren't dealing with circular dependencies and doing some kind of trick to avoid them, then it's better to put all your imports at the top because of the reasons already explained in other answers to this question. And please, when doing this "tricks" include a comment, it's always welcome! :)

4

In addition to the excellent answers already given, it's worth noting that the placement of imports is not merely a matter of style. Sometimes a module has implicit dependencies that need to be imported or initialized first, and a top-level import could lead to violations of the required order of execution.

This issue often comes up in Apache Spark's Python API, where you need to initialize the SparkContext before importing any pyspark packages or modules. It's best to place pyspark imports in a scope where the SparkContext is guaranteed to be available.

3

I do not aspire to provide complete answer, because others have already done this very well. I just want to mention one use case when I find especially useful to import modules inside functions. My application uses python packages and modules stored in certain location as plugins. During application startup, the application walks through all the modules in the location and imports them, then it looks inside the modules and if it finds some mounting points for the plugins (in my case it is a subclass of a certain base class having a unique ID) it registers them. The number of plugins is large (now dozens, but maybe hundreds in the future) and each of them is used quite rarely. Having imports of third party libraries at the top of my plugin modules was a bit penalty during application startup. Especially some thirdparty libraries are heavy to import (e.g. import of plotly even tries to connect to internet and download something which was adding about one second to startup). By optimizing imports (calling them only in the functions where they are used) in the plugins I managed to shrink the startup from 10 seconds to some 2 seconds. That is a big difference for my users.

So my answer is no, do not always put the imports at the top of your modules.

3

It's interesting that not a single answer mentioned parallel processing so far, where it might be REQUIRED that the imports are in the function, when the serialized function code is what is being pushed around to other cores, e.g. like in the case of ipyparallel.

2

There can be a performance gain by importing variables/local scoping inside of a function. This depends on the usage of the imported thing inside the function. If you are looping many times and accessing a module global object, importing it as local can help.

test.py

X=10
Y=11
Z=12
def add(i):
  i = i + 10

runlocal.py

from test import add, X, Y, Z

    def callme():
      x=X
      y=Y
      z=Z
      ladd=add 
      for i  in range(100000000):
        ladd(i)
        x+y+z

    callme()

run.py

from test import add, X, Y, Z

def callme():
  for i in range(100000000):
    add(i)
    X+Y+Z

callme()

A time on Linux shows a small gain

/usr/bin/time -f "\t%E real,\t%U user,\t%S sys" python run.py 
    0:17.80 real,   17.77 user, 0.01 sys
/tmp/test$ /usr/bin/time -f "\t%E real,\t%U user,\t%S sys" python runlocal.py 
    0:14.23 real,   14.22 user, 0.01 sys

real is wall clock. user is time in program. sys is time for system calls.

https://docs.python.org/3.5/reference/executionmodel.html#resolution-of-names

2

Readability

In addition to startup performance, there is a readability argument to be made for localizing import statements. For example take python line numbers 1283 through 1296 in my current first python project:

listdata.append(['tk font version', font_version])
listdata.append(['Gtk version', str(Gtk.get_major_version())+"."+
                 str(Gtk.get_minor_version())+"."+
                 str(Gtk.get_micro_version())])

import xml.etree.ElementTree as ET

xmltree = ET.parse('/usr/share/gnome/gnome-version.xml')
xmlroot = xmltree.getroot()
result = []
for child in xmlroot:
    result.append(child.text)
listdata.append(['Gnome version', result[0]+"."+result[1]+"."+
                 result[2]+" "+result[3]])

If the import statement was at the top of file I would have to scroll up a long way, or press Home, to find out what ET was. Then I would have to navigate back to line 1283 to continue reading code.

Indeed even if the import statement was at the top of the function (or class) as many would place it, paging up and back down would be required.

Displaying the Gnome version number will rarely be done so the import at top of file introduces unnecessary startup lag.

4
  • 1
    What happens if you use ET again later in the script? How would the reader know where to look for the original import if it wasn't at the top of the module? By using a text search/find? The inconvenience of going to the top of the module is outweighed by the ability to quickly identify and maintain/prune all imports a module uses when code changes over time.
    – David Diaz
    Commented Aug 6, 2020 at 16:52
  • @DavidDiaz That program *Multiple Monitor Manager) is now 2,577 lines and still needs more functionality. ET is only used in one place. It is true to your point as soon as an import is used in a second place I move it to the top instead. Commented Aug 8, 2020 at 0:49
  • One could argue that a good text editor would show the import statement for ET on mouseover to avoid this from being a problem... so perhaps readability is limited to text editors which don't do this. AFAIK even GitHub does this on their web UI Commented Feb 16, 2021 at 23:26
  • Also, just on a side note AFAIK most modern IDEs have some sort of Go Back and Go Forward feature which let's you jump back and forth between any of your previous cursor positions. This even works between files, jumping from one file to another in that matter. The shortcuts in VS Code under macOS for those would be ctrl + - for jumping back to the previous and ctrl + shift + - for jumping forward to the next cursor position. Commented Apr 21, 2023 at 11:26
1

I would like to mention a usecase of mine, very similar to those mentioned by @John Millikin and @V.K.:

Optional Imports

I do data analysis with Jupyter Notebook, and I use the same IPython notebook as a template for all analyses. In some occasions, I need to import Tensorflow to do some quick model runs, but sometimes I work in places where tensorflow isn't set up / is slow to import. In those cases, I encapsulate my Tensorflow-dependent operations in a helper function, import tensorflow inside that function, and bind it to a button.

This way, I could do "restart-and-run-all" without having to wait for the import, or having to resume the rest of the cells when it fails.

1

While PEP encourages importing at the top of a module, it isn't an error to import at other levels. That indicates imports should be at the top, however there are exceptions.

It is a micro-optimization to load modules when they are used. Code that is sluggish importing can be optimized later if it makes a sizable difference.

Still, you might introduce flags to conditionally import at as near to the top as possible, allowing a user to use configuration to import the modules they need while still importing everything immediately.

Importing as soon as possible means the program will fail if any imports (or imports of imports) are missing or have syntax errors. If all imports occur at the top of all modules then python works in two steps. Compile. Run.

Built in modules work anywhere they are imported because they are well designed. Modules you write should be the same. Moving around your imports to the top or to their first use can help ensure there are no side effects and the code is injecting dependencies.

Whether you put imports at the top or not, your code should still work when the imports are at the top. So start by importing immediately then optimize as needed.

0

This is a fascinating discussion. Like many others I had never even considered this topic. I got cornered into having to have the imports in the functions because of wanting to use the Django ORM in one of my libraries. I was having to call django.setup() before importing my model classes and because this was at the top of the file it was being dragged into completely non-Django library code because of the IoC injector construction.

I kind of hacked around a bit and ended up putting the django.setup() in the singleton constructor and the relevant import at the top of each class method. Now this worked fine but made me uneasy because the imports weren't at the top and also I started worrying about the extra time hit of the imports. Then I came here and read with great interest everybody's take on this.

I have a long C++ background and now use Python/Cython. My take on this is that why not put the imports in the function unless it causes you a profiled bottleneck. It's only like declaring space for variables just before you need them. The trouble is I have thousands of lines of code with all the imports at the top! So I think I will do it from now on and change the odd file here and there when I'm passing through and have the time.

Not the answer you're looking for? Browse other questions tagged or ask your own question.