71

I have a file called foobar (without .py extension). In the same directory I have another python file that tries to import it:

import foobar

But this only works if I rename the file to foobar.py. Is it possible to import a python module that doesn't have the .py extension?

Update: the file has no extension because I also use it as a standalone script, and I don't want to type the .py extension to run it.

Update2: I will go for the symlink solution mentioned below.

  • 1
    I'm intrigued. Why do you have a python file without the py extension? – Esteban Küber Apr 8 '10 at 16:52
  • 2
    Sometimes it's nice to use python for configuration files (extension as .conf) or to denote a special type of file. In my case, it'd be more of a convenience for an Administrator. – NuclearPeon Aug 2 '13 at 3:01
  • 1
    I have a file with configuration that is used both as a python file and as a bash script. I gave it a pysh extension... – osa Jul 11 '14 at 21:43
  • If that is configuration related things, I recommend using ConfigParser. wiki.python.org/moin/ConfigParserExamples – Chemical Programmer Nov 23 '15 at 7:53
  • 2
    @voyager One reason is python scripts with .cgi extensions instead of .py extension – repzero Jun 15 '16 at 1:49
50

You can use the imp.load_source function (from the imp module), to load a module dynamically from a given file-system path.

import imp
foobar = imp.load_source('foobar', '/path/to/foobar')

This SO discussion also shows some interesting options.

| improve this answer | |
  • Fixed. [it is more constructive to suggest an Edit, though] – Eli Bendersky May 14 '15 at 14:02
  • 2
    What is the use of the first argument 'foobar' if it is assigned in the return value? – Anmol Singh Jaggi Apr 30 '16 at 14:13
  • 2
    @AnmolSinghJaggi It sets the module __name__ property; normally that's determined from the filename, but since you're using a non-standard filename (which might not even contain any valid python identifier at all), you have to specify the module name. The variable name in which you store a reference to the created module object is irrelevant, much as if you import foo.bar as baz the module referenced by the variable baz will still have its original __name__. – Ben Apr 26 '17 at 2:40
  • 3
    This has been deprecated since 3.4. Any idea how to import from a file without the .py extension in 3.4+? – exhuma Oct 6 '17 at 9:42
  • 2
    For Python 3.4+: this answer is more exact: stackoverflow.com/questions/2601047/… – tres.14159 Oct 29 '18 at 12:48
20

Here is a solution for Python 3.4+:

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader 

spec = spec_from_loader("foobar", SourceFileLoader("foobar", "/path/to/foobar"))
foobar = module_from_spec(spec)
spec.loader.exec_module(foobar)

Using spec_from_loader and explicitly specifying a SourceFileLoader will force the machinery to load the file as source, without trying to figure out the type of the file from the extension. This means that you can load the file even though it is not listed in importlib.machinery.SOURCE_SUFFIXES.

If you want to keep importing the file by name after the first load, add the module to sys.modules:

sys.modules['foobar'] = foobar
| improve this answer | |
  • 2
    That module needs a foobar = importlib.nice_import('foobar') helper desperately. – Ciro Santilli 郝海东冠状病六四事件法轮功 Sep 8 '18 at 8:08
  • @Ciro. It already has that. I don't think that /some/arbitrary/file.weird_extension qualifies as "nice". That being said, I've started using python code for all my configuration files once I discovered this. It's just so convenient. – Mad Physicist Sep 8 '18 at 14:04
  • What if I want to import * ? – Alex Harvey Feb 27 at 4:51
  • 1
    @AlexHarvey. This gives you a module object. You can do something like globals().update(foobar.__dict__) or so, but I would recommend against it. – Mad Physicist Feb 27 at 13:24
16

Like others have mentioned, you could use imp.load_source, but it will make your code more difficult to read. I would really only recommend it if you need to import modules whose names or paths aren't known until run-time.

What is your reason for not wanting to use the .py extension? The most common case for not wanting to use the .py extension, is because the python script is also run as an executable, but you still want other modules to be able to import it. If this is the case, it might be beneficial to move functionality into a .py file with a similar name, and then use foobar as a wrapper.

| improve this answer | |
  • 14
    Or instead of wrapping, just symlink foobar.py to foobar (assuming you aren't on Windows) – whaley Apr 8 '10 at 18:31
  • @whaley, yeah, that would be much cleaner. You could use a .bat for windows to accomplish the same thing. – user297250 Apr 8 '10 at 22:45
  • I've got a neat use case - a readme file, with examples in it, which I'd like doctest to validate. I'm hoping to make a doctest markdown doc that works... – Danny Staple Nov 24 '11 at 23:05
  • And the answer is (for that use case) - use doctest.loadfile! – Danny Staple Nov 24 '11 at 23:10
  • The issue with wrappers is that if someone naively copies just the wrapper to a bin/ directory, the program won't work when run from the path. – cjs Nov 22 '17 at 1:14
14

imp.load_source(module_name, path) should do or you can do the more verbose imp.load_module(module_name, file_handle, ...) route if you have a file handle instead

| improve this answer | |
6

importlib helper function

Here is a convenient, ready-to-use helper to replace imp, with an example, based on what was mentioned at: https://stackoverflow.com/a/43602645/895245

main.py

#!/usr/bin/env python3

import os
import importlib

def import_path(path):
    module_name = os.path.basename(path).replace('-', '_')
    spec = importlib.util.spec_from_loader(
        module_name,
        importlib.machinery.SourceFileLoader(module_name, path)
    )
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    sys.modules[module_name] = module
    return module

notmain = import_path('not-main')
print(notmain)
print(notmain.x)

not-main

x = 1

Run:

python3 main.py

Output:

<module 'not_main' from 'not-main'>
1

I replace - with _ because my importable Python executables without extension have hyphens. This is not mandatory, but produces better module names.

This pattern is also mentioned in the docs at: https://docs.python.org/3.7/library/importlib.html#importing-a-source-file-directly

I ended up moving to it because after updating to Python 3.7, import imp prints:

DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

and I don't know how to turn that off.

Tested in Python 3.7.3.

| improve this answer | |
1

If you install the script with package manager (deb or alike) another option would be to use setuptools:

"...there’s no easy way to have a script’s filename match local conventions on both Windows and POSIX platforms. For another, you often have to create a separate file just for the “main” script, when your actual “main” is a function in a module somewhere... setuptools fixes all of these problems by automatically generating scripts for you with the correct extension, and on Windows it will even create an .exe file..."

https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation

| improve this answer | |
0

I have tried all the suggestions above and can not find a solution for a really simple case where I want to pytest a cli script that has no .py extension.

Why does spec.loader.exec_module() raise a IsADirectoryError on "."?

I have two files in a directory:

% ls

cli_with_no_dot_py      test_cli_with_no_dot_py.py

% cat cli_with_no_dot_py

#!/usr/bin/env python3

cool_thing = True
print("cool_thing is", cool_thing)

% ./cli_with_no_dot_py

cool_thing is True

% cat test_cli_with_no_dot_py.py

#!/usr/bin/env python3

from importlib.util import spec_from_loader, module_from_spec
from importlib.machinery import SourceFileLoader
spec = spec_from_loader("cli_with_no_dot_py",
    SourceFileLoader("cli_with_no_dot_py", "."))
print("spec:", spec)
cli_with_no_dot_py = module_from_spec(spec)
print("cli_with_no_dot_py):", cli_with_no_dot_py)
print("dir(cli_with_no_dot_py:", dir(cli_with_no_dot_py))
print("spec.loader:", spec.loader)
spec.loader.exec_module(cli_with_no_dot_py)  # !!! IsADirectoryError !!!

def test_cool_thing():
    print("cli_with_no_dot_py.cool_thing is", cli_with_no_dot_py.cool_thing)
    assert cli_with_no_dot_py.cool_thing

% pytest

======================================================================== test session starts ========================================================================
platform darwin -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /Users/cclauss/Python/pytest_a_cli_tool
collected 0 items / 1 error

============================================================================== ERRORS ===============================================================================
____________________________________________________________ ERROR collecting test_cli_with_no_dot_py.py ____________________________________________________________
test_cli_with_no_dot_py.py:11: in <module>
    spec.loader.exec_module(cli_with_no_dot_py)
<frozen importlib._bootstrap_external>:779: in exec_module
    ???
<frozen importlib._bootstrap_external>:915: in get_code
    ???
<frozen importlib._bootstrap_external>:972: in get_data
    ???
E   IsADirectoryError: [Errno 21] Is a directory: '.'
-------------------------------------------------------------------------- Captured stdout --------------------------------------------------------------------------
spec: ModuleSpec(name='cli_with_no_dot_py', loader=<_frozen_importlib_external.SourceFileLoader object at 0x10e106f70>, origin='.')
cli_with_no_dot_py: <module 'cli_with_no_dot_py' from '.'>
dir(cli_with_no_dot_py): ['__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__']
spec.loader: <_frozen_importlib_external.SourceFileLoader object at 0x10e106f70>
====================================================================== short test summary info ======================================================================
ERROR test_cli_with_no_dot_py.py - IsADirectoryError: [Errno 21] Is a directory: '.'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================= 1 error in 0.09s ==========================================================================
| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.