'Builtin python's __import__ vs imp.load_module: ValueError: Attempted relative import beyond toplevel package

I have a piece of code that dynamically imports a bunch of subpackages - package structure:

main_package/
    code_below_is_here.py
    game/
        __init__.py
        game1/
            __init__.py
            constants.py
            records.py
            ...
        game2/
            __init__.py
            constants.py
            records.py
            ...

using the __import__ function (disclaimer: I did not write that code I 'm just the current maintainer):

import pkgutil
import game as game_init# <----- I import the game/ package
# Detect the known games
for importer,modname,ispkg in pkgutil.iter_modules(game_init.__path__):
    if not ispkg: continue # game support modules are packages
    # Equivalent of "from game import <modname>"
    try:
        module = __import__('game',globals(),locals(),[modname],-1)
    except ImportError:
        continue

This works just fine.

Now I was trying to use imp.load_module to do the same thing (trying to answer another question of mine):

     for importer, modname, ispkg in pkgutil.iter_modules(game_init.__path__):
         if not ispkg: continue # game support modules are packages
         # Equivalent of "from game import <modname>"
+        init_file, pathname, description = imp.find_module(modname, game_init.__path__)
         try:
-            module = __import__('game',globals(),locals(),[modname],-1)
+            module = imp.load_module(modname, init_file, pathname, description)
         except ImportError:
             deprint(u'Error in game support module:', modname, traceback=True)
             continue
+        finally:
+            if init_file is not None: init_file.close()

But this fails with:

Traceback (most recent call last):
...
  File "C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\bash\bush.py", line 64, in _supportedGames
    module = imp.load_module(modname, init_file, pathname, description)
  File "C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\bash\game\fallout4\__init__.py", line 32, in <module>
    from .records import MreHeader, MreLvli, MreLvln
  File "C:\Dropbox\eclipse_workspaces\python\wrye-bash\Mopy\bash\game\fallout4\records.py", line 26, in <module>
    from ..skyrim.records import MelBounds, MreLeveledList
ValueError: Attempted relative import beyond toplevel package

I guess that the answer is somnewhere in __import__'s docs, where we read:

The function imports the module name, potentially using the given globals and locals to determine how to interpret the name in a package context. The fromlist gives the names of objects or submodules that should be imported from the module given by name. The standard implementation does not use its locals argument at all, and uses its globals only to determine the package context of the import statement.

So:

  • is it possible to call imp.load_module with a different set of arguments so it is made aware that the top level package is, indeed, not game/gameX ?

Related:



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source