31

I build quite complex python apps, often with Django. To simplify inter-application interfaces I sometimes use service.py modules that abstract away from the models.

As these 'aggregate functionality', they frequently end up with circular imports which are easily eliminated by placing the import statements inside the service functions.

Is there a significant performance or memory cost associated with generally moving imports as close to their point of use as possible? For example, if I only use a particular imported name in one function in a file, it seems natural to place the import in that particular function rather than at the top of the file in its conventional place.

This issue is subtly different to this question because each import is in the function namespace.

2
  • 2
    @KevinBrown: I think the questions are fairly close, but not really duplicates. That one is about 'safety' and this one is about exploring performance penalties.
    – code_dredd
    Aug 22, 2015 at 6:14
  • none of the answers actually answers the question! :( They all are talking about import Whole module vs from XX import YYY Apr 25, 2019 at 20:09

4 Answers 4

42

The point at which you import a module is not expected to cause a performance penalty, if that's what you're worried about. Modules are singletons and will not be imported every single time an import statement is encountered. However, how you do the import, and subsequent attribute lookups, does have an impact.

For example, if you import math and then every time you need to use the sin(...) function you have to do math.sin(...), this will generally be slower than doing from math import sin and using sin(...) directly as the system does not have to keep looking up the function name within the module.

This lookup-penalty applies to anything that is accessed using the dot . and will be particularly noticeable in a loop. It's therefore advisable to get a local reference to something you might need to use/invoke frequently in a performance critical loop/section.

For example, using the original import math example, right before a critical loop, you could do something like this:

# ... within some function
sin = math.sin
for i in range(0, REALLY_BIG_NUMBER):
    x = sin(i)   # faster than:  x = math.sin(x)
    # ...

This is a trivial example, but note that you could do something similar with methods on other objects (e.g. lists, dictionaries, etc).

I'm probably a bit more concerned about the circular imports you mention. If your intention is to "fix" circular imports by moving the import statements into more "local" places (e.g. within a specific function, or block of code, etc) you probably have a deeper issue that you need to address.

Personally, I'd keep the imports at the top of the module as it's normally done. Straying away from that pattern for no good reason is likely to make your code more difficult to go through because the dependencies of your module will not be immediately apparent (i.e. there're import statements scattered throughout the code instead of in a single location).

It might also make the circular dependency issue you seem to be having more difficult to debug and easier to fall into. After all, if the module is not listed above, someone might happily think your module A has no dependency on module B and then up adding an import A in B when A already has import B hidden in some deep dark corner.


Benchmark Sample

Here's a benchmark using the lookup notation:

>>> timeit('for i in range(0, 10000): x = math.sin(i)', setup='import math', number=50000)
89.7203312900001

And another benchmark not using the lookup notation:

>>> timeit('for i in range(0, 10000): x = sin(i)', setup='from math import sin', number=50000)
78.27029322999988

Here there's a 10+ second difference.

Note that your gain depends on how much time the program spends running this code --i.e. a performance critical section instead of sporadic function calls.

4
  • 1
    Just out of interest. Do you have a source/timings showing the difference between those calls? Aug 22, 2015 at 1:47
  • @PaulThompson: Updated my response and included benchmark results for it.
    – code_dredd
    Aug 22, 2015 at 6:12
  • The circular dependencies are a concern but they come from the fact that Python does not offer an easy way to separate methods on a specific class... using mixins might be a way out in the long term.
    – Paul Whipp
    Aug 24, 2015 at 0:03
  • @PaulWhipp: Mixins are good alternatives to multiple inheritance (among others), but not to avoid circular deps. Consider whether the modules are related enough to justify being merged into one. Also make sure your higher-level modules depend on lower-level ones and not the other way around. E.g. see how Android Applications depend on the Application Framework, but not the other way around in the Android Architecture Diagram.
    – code_dredd
    Aug 24, 2015 at 3:47
9

See this question.

Basically whenever you import a module, if it's been imported before it will use a cached value.

This means that the performance will be hit the first time that the module is loaded, but once it's been loaded it will cache the values for future calls to it.

0

As ray said, importing specific functions is (slightly faster) 1.62852311134 for sin() 1.89815092087 for math.sin() using the following code

from time import time
sin=math.sin
t1=time()
for i in xrange(10000000):
x=sin(i)
t2=time()
for i in xrange(10000000):
    z=math.sin(i)
t3=time()
print (t2-t1)
print (t3-t2)
3
  • You should use the timeit module if you intend to do performance benchmarks in Python. See my updated response :)
    – code_dredd
    Aug 22, 2015 at 6:19
  • What is the difference? I understand that for more complex things, the compiler can add optimizations that will mess up the results, but in something as simple as this, is there any real difference? Aug 22, 2015 at 14:22
  • See the response to this question.
    – code_dredd
    Aug 22, 2015 at 19:50
0

As per timeit, there is a significant cost to an import statement, even when the module is already imported in the same namespace:

$ python -m timeit -s 'import sys
def foo():
    import sys
    assert sys is not None
' -- 'foo()'
500000 loops, best of 5: 824 nsec per loop
$ python -m timeit -s 'import sys
def foo():
    assert sys is not None
' -- 'foo()'
2000000 loops, best of 5: 96.3 nsec per loop

(Timing figures from Python 3.10.6 on Termux running on a phone.)

Instead of imports within functions, I've found that I can take advantage of Python's support for partially initialized modules and do a "tail import", pushing the import statement to the very bottom of the file (with a # isort:skip to get isort to leave it alone). This allows circular imports as long as the tail import is not required at module or class level and only at function or method level.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.