9

I would like to know if the following behavior is expected or a bug. I'm using CPython2.7

Create a file x.py

def funcA():
    print "funcA of x.py"
def funcB():
    print "funcB of x.py"

Create a file y.py

def funcB():
    print "funcB of y.py"

Create a file test.py

import sys, imp
# load x.py as fff
m = imp.load_source('fff', 'x.py')
print dir(m)
print sys.modules.get('fff')
# load y.py as fff
m = imp.load_source('fff', 'y.py')
print dir(m)    
print sys.modules.get('fff')

# import and exec func
import fff
fff.funcA()
fff.funcB()
print dir(fff)

The result

['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']
<module 'fff' from 'x.py'>
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']
<module 'fff' from 'y.py'>
funcA of x.py
funcB of y.py
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'funcA', 'funcB']

My expectation was that the 2nd imp.load_source would completely replace the module x.py with y.py. In fact sys.modules.get('fff') shows <module 'fff' from 'y.py'> but the resulting module was like a mix of x.py and y.py, and the latter has the precedence.

Is this expected or a bug?

EDIT: my test code had a typo. updated the result.

1 Answer 1

15

This is an expected behavior.
See http://docs.python.org/2/library/imp.html

imp.load_source(name, pathname[, file])

Load and initialize a module implemented as a Python source file and return its module object. If the module was already initialized, it will be initialized again. The name argument is used to create or access a module object.

Since your second module has the same name as the first module, it won't replace the first one, but will be merged into the first one.

The source code gives us the same answer.
imp is a built-in module, defined in import.c.
Let's look at at the definition of load_source

static PyObject *
load_source_module(char *name, char *pathname, FILE *fp)
{
    ......
    m = PyImport_ExecCodeModuleEx(name, (PyObject *)co, pathname);
    Py_DECREF(co);

    return m;
}

It's just a wrapper for PyImport_ExecCodeModuleEx.

PyObject *
PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
{
    PyObject *modules = PyImport_GetModuleDict();
    PyObject *m, *d, *v;

    m = PyImport_AddModule(name);
    ......
    d = PyModule_GetDict(m);
    ......
    v = PyEval_EvalCode((PyCodeObject *)co, d, d);
    ......
}

Now, we just need to focus on PyImport_AddModule. Python uses it to obtain a module object. Your parsed source file will be put into this module object.

PyObject *
PyImport_AddModule(const char *name)
{
    PyObject *modules = PyImport_GetModuleDict();
    PyObject *m;

    if ((m = PyDict_GetItemString(modules, name)) != NULL &&
        PyModule_Check(m))
        return m;
    m = PyModule_New(name);
    if (m == NULL)
        return NULL;
    if (PyDict_SetItemString(modules, name, m) != 0) {
        Py_DECREF(m);
        return NULL;
    }
    Py_DECREF(m); /* Yes, it still exists, in modules! */

    return m;
}

Finally, we find the answer. Given a name, if some module already has this name, namely, name in sys.modules, Python won't create a new module, but will reuse that module.

3
  • 3
    Thank you very much for looking all the way into import.c. I wonder what's the good use case of this. The document could be more explicit about the merger. Other functions in the same module such as load_module and reload has a much better documentation. Feb 26, 2013 at 17:41
  • 2
    Is there a way that the modules are not merged but replaced?
    – foobar
    Dec 29, 2013 at 20:51
  • 1
    yes, always del sys.modules['name of the module'] before imp.load_source. Jun 17, 2014 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.