3

I'm trying to create an auto_import function which is part of a library: the purpose of this to avoid listing from .x import y many times in __init__ files, only do something this import lib; lib.auto_import(__file__) <- this would search for python files in that folder where the __init__ is present and would import all stuff by exec statement (i.e. exec('from .x import abc')).

My problem is that, somehow the 'from' statement always tries to import .x from lib directory, even if I change the cwd to the directory where the actual __init__ file is placed... How should I solve this? How should I change the search dir for from . statement?

Structure:

$ ls -R
.:
app.py  lib  x

./lib:
__init__.py  auto_import.py

./x:
__init__.py  y

./x/y:
__init__.py  y.py

e.g.: ./x/y/__init__.py contains import lib; lib.auto_import(__file__) auto_import is checking for files in dir of __file__ and import them with exec('from .{} import *') (but this from . is always the lib folder and not the dir of __file__, and that is my question, how to change this to dir of __file__ Of course the whole stuff is imported in app.py like:

import x
print(x.y) 

Thanks

EDIT1: final auto_import (globals() / gns cannot be avoided )

import os, sys, inspect

def auto_import(gns):
  current_frame = inspect.currentframe()
  caller_frame = inspect.getouterframes(current_frame)[1]
  src_file = caller_frame[1]
  for item in os.listdir(os.path.dirname(src_file)):
    item = item.split('.py')[0]

    if item in ['__init__', '__pycache__']:
      continue

    gns.update(__import__(item, gns, locals(), ['*'], 1).__dict__)
7
  • Side note: although I have not seen this till the end yet, you might find some useful things here.
    – quapka
    May 13, 2016 at 21:12
  • Can you give an example of the file/folder structure of your lib? It's quite difficult to follow your question.
    – gdlmx
    May 13, 2016 at 21:20
  • ok, added the structure
    – vpas
    May 13, 2016 at 21:22
  • Now the question is clearer. The problem of your approach is that auto_import is defined in lib/auto_import.py so the context for exec('from .x import *') is always lib/. Even though you manage to fix the path problem, lib.auto_import(__file__) will not import anything to the namespace of lib.x.y, because the function locates in another module.
    – gdlmx
    May 13, 2016 at 21:37
  • 1
    No, there is a better way to do that, I will test it and post an answer later
    – gdlmx
    May 13, 2016 at 21:42

2 Answers 2

1

The problem of your approach is that auto_import is defined in lib/auto_import.py so the context for exec('from .x import *') is always lib/. Even though you manage to fix the path problem, lib.auto_import(__file__) will not import anything to the namespace of lib.x.y, because the function locates in another module.

Use the built-in function __import__

Here is the auto_import script:

myimporter.py

# myimporter.py
def __import_siblings__(gns, lns={}):
  for name in find_sibling_names(gns['__file__']):
    gns.update((k,v) for k,v in __import__(name, gns,lns).__dict__.items() if not k.startswith('_'))
import re,os
def find_sibling_names(filename):
  pyfp = re.compile(r'([a-zA-Z]\w*)\.py$')
  files = (pyfp.match(f) for f in os.listdir(os.path.dirname(filename)))
  return set(f.group(1) for f in files if f)

Inside your lib/x/y/__init__.py

#lib/x/y/__init__.py
from myimporter import __import_siblings__
__import_siblings__(globals())

Let's say you have a dummy module that need to be imported to y:

#lib/x/y/dummy.py
def hello():
  print 'hello'

Test it:

import x.y
x.y.hello()

Please be aware that from lib import * is usually a bad habit because of namespace pollution. Use it with caution.

Refs: 1 2

4
  • Working! And almost beautiful, but is it possible to somehow avoid the globals(), locals() pass always to the importer function? I mean, it would be more beautiful if those "code duplications" would not be there always, instead somehow the import function would "calculate" them
    – vpas
    May 13, 2016 at 22:59
  • As I see the critical is the globals variable as this still gives fine output: globals().update(import__(item, gns, locals(), ['*'], 1).__dict) but if i replace that gns with "globals()" then it cannot import
    – vpas
    May 13, 2016 at 23:01
  • ok... maybe this is also applicable for the file variable, but still, im courius if you have any idea about that :)
    – vpas
    May 13, 2016 at 23:04
  • gns is important for the __import__ function because it uses this namespace to determine the searching path. There is some tricks to avoid passing globals() using CPython stack_frame, but that is not standard and only works with CPython.
    – gdlmx
    May 13, 2016 at 23:06
0

Use sys module With this folder structure:

RootDir
    |-- module.py
    |--ChildDir
        |-- main.py

Now in main.py you can do

import sys
sys.path.append('..')
import module

A believe there are other hacks, but this is the one I know of and works for my purpose. I am not sure, whether it is the best option to go for some kind of auto_import stuff though.

2
  • I don't need another solution, I need the correction for my solution, thanks ;)
    – vpas
    May 13, 2016 at 21:10
  • This is not safe because it affect the python interpreter and may accidentally break other libraries.
    – gdlmx
    May 13, 2016 at 21:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.