What will happen if two modules import each other?
To generalize the problem, what about the cyclic imports in Python?
What will happen if two modules import each other?
To generalize the problem, what about the cyclic imports in Python?
If you do import foo
(inside bar.py
) and import bar
(inside foo.py
), it will work fine. By the time anything actually runs, both modules will be fully loaded and will have references to each other.
The problem is when instead you do from foo import abc
(inside bar.py
) and from bar import xyz
(inside foo.py
). Because now each module requires the other module to already be imported (so that the name we are importing exists) before it can be imported.
from foo import *
and from bar import *
will also work fine.
from x import y
, and yet still gets the circular import error
Jun 30, 2014 at 14:09
import
statement is executed. So it won't error out but you may not get all the variables you expect.
from foo import *
and from bar import *
, everything executed in the foo
is in the initializing phase of bar
, and the actual functions in bar
has not yet been defined...
Jan 18, 2017 at 2:37
There was a really good discussion on this over at comp.lang.python last year. It answers your question pretty thoroughly.
Imports are pretty straightforward really. Just remember the following:
'import' and 'from xxx import yyy' are executable statements. They execute when the running program reaches that line.
If a module is not in sys.modules, then an import creates the new module entry in sys.modules and then executes the code in the module. It does not return control to the calling module until the execution has completed.
If a module does exist in sys.modules then an import simply returns that module whether or not it has completed executing. That is the reason why cyclic imports may return modules which appear to be partly empty.
Finally, the executing script runs in a module named __main__, importing the script under its own name will create a new module unrelated to __main__.
Take that lot together and you shouldn't get any surprises when importing modules.
Cyclic imports terminate, but you need to be careful not to use the cyclically-imported modules during module initialization.
Consider the following files:
a.py:
print "a in"
import sys
print "b imported: %s" % ("b" in sys.modules, )
import b
print "a out"
b.py:
print "b in"
import a
print "b out"
x = 3
If you execute a.py, you'll get the following:
$ python a.py
a in
b imported: False
b in
a in
b imported: True
a out
b out
a out
On the second import of b.py (in the second a in
), the Python interpreter does not import b
again, because it already exists in the module dict.
If you try to access b.x
from a
during module initialization, you will get an AttributeError
.
Append the following line to a.py
:
print b.x
Then, the output is:
$ python a.py
a in
b imported: False
b in
a in
b imported: True
a out
Traceback (most recent call last):
File "a.py", line 4, in <module>
import b
File "/home/shlomme/tmp/x/b.py", line 2, in <module>
import a
File "/home/shlomme/tmp/x/a.py", line 7, in <module>
print b.x
AttributeError: 'module' object has no attribute 'x'
This is because modules are executed on import and at the time b.x
is accessed, the line x = 3
has not be executed yet, which will only happen after b out
.
__name__
instead of 'a'
. At the beginning, I was totally confused why a file would be executed twice.
As other answers describe this pattern is acceptable in python:
def dostuff(self):
from foo import bar
...
Which will avoid the execution of the import statement when the file is imported by other modules. Only if there is a logical circular dependency, this will fail.
Most Circular Imports are not actually logical circular imports but rather raise ImportError
errors, because of the way import()
evaluates top level statements of the entire file when called.
These ImportErrors
can almost always be avoided if you positively want your imports on top:
Consider this circular import:
# profiles/serializers.py
from images.serializers import SimplifiedImageSerializer
class SimplifiedProfileSerializer(serializers.Serializer):
name = serializers.CharField()
class ProfileSerializer(SimplifiedProfileSerializer):
recent_images = SimplifiedImageSerializer(many=True)
# images/serializers.py
from profiles.serializers import SimplifiedProfileSerializer
class SimplifiedImageSerializer(serializers.Serializer):
title = serializers.CharField()
class ImageSerializer(SimplifiedImageSerializer):
profile = SimplifiedProfileSerializer()
From David Beazleys excellent talk Modules and Packages: Live and Let Die! - PyCon 2015, 1:54:00
, here is a way to deal with circular imports in python:
try:
from images.serializers import SimplifiedImageSerializer
except ImportError:
import sys
SimplifiedImageSerializer = sys.modules[__package__ + '.SimplifiedImageSerializer']
This tries to import SimplifiedImageSerializer
and if ImportError
is raised, because it already is imported, it will pull it from the importcache.
PS: You have to read this entire post in David Beazley's voice.
Module a.py :
import b
print("This is from module a")
Module b.py
import a
print("This is from module b")
Running "Module a" will output:
>>>
'This is from module a'
'This is from module b'
'This is from module a'
>>>
It output this 3 lines while it was supposed to output infinitival because of circular importing. What happens line by line while running"Module a" is listed here:
import b
. so it will visit module bimport a
. so it will visit module aimport b
but note that this line won't be executed again anymore, because every file in python execute an import line just for once, it does not matter where or when it is executed. so it will pass to the next line and print "This is from module a"
."This is from module b"
"This is from module a"
and program will be finished.a.py
, when executed as a script, will be named "module __main__
", not "module a
". So when it gets to b
and encounters import a
, now it will import the same file under a different module name, right? What happens when neither are the __main__
script?
Oct 9, 2021 at 3:38
I got an example here that struck me!
foo.py
import bar
class gX(object):
g = 10
bar.py
from foo import gX
o = gX()
main.py
import foo
import bar
print "all done"
At the command line: $ python main.py
Traceback (most recent call last):
File "m.py", line 1, in <module>
import foo
File "/home/xolve/foo.py", line 1, in <module>
import bar
File "/home/xolve/bar.py", line 1, in <module>
from foo import gX
ImportError: cannot import name gX
bar
and foo
both must use gX
, the 'cleanest' solution is to put gX
in another module and have both foo
and bar
import that module. (cleanest in the sense that there are no hidden semantic dependencies.)
Dec 17, 2013 at 20:32
bar
can't even find gX
in the foo. the circular import is fine by itself, but it's just that gX
is not defined when it's imported.
Jan 18, 2017 at 3:31
To my surprise, no one has mentioned cyclic imports caused by type hints yet.
If you have cyclic imports only as a result of type hinting, they can be avoided in a clean manner.
Consider main.py
which makes use of exceptions from another file:
from src.exceptions import SpecificException
class Foo:
def __init__(self, attrib: int):
self.attrib = attrib
raise SpecificException(Foo(5))
And the dedicated exception class exceptions.py
:
from src.main import Foo
class SpecificException(Exception):
def __init__(self, cause: Foo):
self.cause = cause
def __str__(self):
return f'Expected 3 but got {self.cause.attrib}.'
This will trivially raise an ImportError
as main.py
imports exception.py
and vice versa through Foo
and SpecificException
.
Because Foo
is only required in exceptions.py
during type checking, we can safely make its import conditional using the TYPE_CHECKING
constant from the typing module. The constant is only True
during type checking, which allows us to conditionally import Foo
and thereby avoid the circular import error.
In Python 3.6, with the use of forward references:
from typing import TYPE_CHECKING
if TYPE_CHECKING: # Only imports the below statements during type checking
from src.main import Foo
class SpecificException(Exception):
def __init__(self, cause: 'Foo'): # The quotes make Foo a forward reference
self.cause = cause
def __str__(self):
return f'Expected 3 but got {self.cause.attrib}.'
In Python 3.7+, postponed evaluation of annotations (introduced in PEP 563) allows 'normal' types to be used instead of forward references:
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING: # Only imports the below statements during type checking
from src.main import Foo
class SpecificException(Exception):
def __init__(self, cause: Foo): # Foo can be used in type hints without issue
self.cause = cause
def __str__(self):
return f'Expected 3 but got {self.cause.attrib}.'
In Python 3.11+, from __future__ import annotations
is active by default and can therefore be omitted.
This answer is based on Yet another solution to dig you out of a circular import hole in Python by Stefaan Lippens.
There are a lot of great answers here. While there are usually quick solutions to the problem, some of which feel more pythonic than others, if you have the luxury of doing some refactoring, another approach is to analyze the organization of your code, and try to remove the circular dependency. You may find, for example, that you have:
File a.py
from b import B
class A:
@staticmethod
def save_result(result):
print('save the result')
@staticmethod
def do_something_a_ish(param):
A.save_result(A.use_param_like_a_would(param))
@staticmethod
def do_something_related_to_b(param):
B.do_something_b_ish(param)
File b.py
from a import A
class B:
@staticmethod
def do_something_b_ish(param):
A.save_result(B.use_param_like_b_would(param))
In this case, just moving one static method to a separate file, say c.py
:
File c.py
def save_result(result):
print('save the result')
will allow removing the save_result
method from A, and thus allow removing the import of A from a in b:
Refactored File a.py
from b import B
from c import save_result
class A:
@staticmethod
def do_something_a_ish(param):
A.save_result(A.use_param_like_a_would(param))
@staticmethod
def do_something_related_to_b(param):
B.do_something_b_ish(param)
Refactored File b.py
from c import save_result
class B:
@staticmethod
def do_something_b_ish(param):
save_result(B.use_param_like_b_would(param))
In summary, if you have a tool (e.g. pylint or PyCharm) that reports on methods that can be static, just throwing a staticmethod
decorator on them might not be the best way to silence the warning. Even though the method seems related to the class, it might be better to separate it out, especially if you have several closely related modules that might need the same functionality and you intend to practice DRY principles.
I completely agree with pythoneer's answer here. But I have stumbled on some code that was flawed with circular imports and caused issues when trying to add unit tests. So to quickly patch it without changing everything you can resolve the issue by doing a dynamic import.
# Hack to import something without circular import issue
def load_module(name):
"""Load module using imp.find_module"""
names = name.split(".")
path = None
for name in names:
f, path, info = imp.find_module(name, path)
path = [path]
return imp.load_module(name, f, path[0], info)
constants = load_module("app.constants")
Again, this isn't a permanent fix but may help someone that wants to fix an import error without changing too much of the code.
Cheers!
Circular imports can be confusing because import does two things:
The former is done only once, while the latter at each import statement. Circular import creates situation when importing module uses imported one with partially executed code. In consequence it will not see objects created after import statement. Below code sample demonstrates it.
Circular imports are not the ultimate evil to be avoided at all cost. In some frameworks like Flask they are quite natural and tweaking your code to eliminate them does not make the code better.
main.py
print 'import b'
import b
print 'a in globals() {}'.format('a' in globals())
print 'import a'
import a
print 'a in globals() {}'.format('a' in globals())
if __name__ == '__main__':
print 'imports done'
print 'b has y {}, a is b.a {}'.format(hasattr(b, 'y'), a is b.a)
b.by
print "b in, __name__ = {}".format(__name__)
x = 3
print 'b imports a'
import a
y = 5
print "b out"
a.py
print 'a in, __name__ = {}'.format(__name__)
print 'a imports b'
import b
print 'b has x {}'.format(hasattr(b, 'x'))
print 'b has y {}'.format(hasattr(b, 'y'))
print "a out"
python main.py output with comments
import b
b in, __name__ = b # b code execution started
b imports a
a in, __name__ = a # a code execution started
a imports b # b code execution is already in progress
b has x True
b has y False # b defines y after a import,
a out
b out
a in globals() False # import only adds a to main global symbol table
import a
a in globals() True
imports done
b has y True, a is b.a True # all b objects are available
I solved the problem the following way, and it works well without any error.
Consider two files a.py
and b.py
.
I added this to a.py
and it worked.
if __name__ == "__main__":
main ()
import b
y = 2
def main():
print ("a out")
print (b.x)
if __name__ == "__main__":
main ()
import a
print ("b out")
x = 3 + a.y
The output I get is
>>> b out
>>> a out
>>> 5
Suppose you are running a test python file named request.py
In request.py, you write
import request
so this also most likely a circular import.
Solution:
Just change your test file to another name such as aaa.py
, other than request.py
.
Do not use names that are already used by other libs.
Ok, I think I have a pretty cool solution.
Let's say you have file a
and file b
.
You have a def
or a class
in file b
that you want to use in module a
, but you have something else, either a def
, class
, or variable from file a
that you need in your definition or class in file b
.
What you can do is, at the bottom of file a
, after calling the function or class in file a
that is needed in file b
, but before calling the function or class from file b
that you need for file a
, say import b
Then, and here is the key part, in all of the definitions or classes in file b
that need the def
or class
from file a
(let's call it CLASS
), you say from a import CLASS
This works because you can import file b
without Python executing any of the import statements in file b
, and thus you elude any circular imports.
For example:
class A(object):
def __init__(self, name):
self.name = name
CLASS = A("me")
import b
go = B(6)
go.dostuff
class B(object):
def __init__(self, number):
self.number = number
def dostuff(self):
from a import CLASS
print "Hello " + CLASS.name + ", " + str(number) + " is an interesting number."
Voila.
from a import CLASS
doesn't actually skip executing all the code in a.py. This is what really happens: (1) All the code in a.py gets run as a special module "__main__". (2) At import b
, the top-level code in b.py gets run (defining class B) and then control returns to "__main__". (3) "__main__" eventually passes control to go.dostuff()
. (4) when dostuff() comes to import a
, it runs all the code in a.py again, this time as the module "a"; then it imports the CLASS object from the new module "a". So actually, this would work equally well if you used import a
anywhere in b.py.
Jun 4, 2015 at 7:28