← Python

Modules, Packages & Imports

← Modules, Packages & Imports

Select a question from the sidebar

10 questions in this topic

Question 1 / 10

How does Python's import system work, and what happens when you import a module?

When you execute import module, Python follows this process:

Check sys.modules cache: If the module was already imported, return the cached version. This prevents re-execution and ensures singletons.
Find the module: Search sys.path (list of directories) and finders in sys.meta_path for the module.
Load the module: Use the appropriate loader (file-based, built-in, frozen, etc.) to read the module's code.
Create module object: Create a new types.ModuleType object.
Add to sys.modules: Cache the module object BEFORE executing the code (important for circular imports).
Execute module code: Run the module's top-level code in the module's namespace.
Bind the name: Bind the module name in the importing module's namespace.

The search order in sys.path is: current directory, PYTHONPATH environment variable, standard library, site-packages (third-party).

Pseudo Code

import foo:
1. if 'foo' in sys.modules: return cached
2. Find foo.py in sys.path
3. Create module object
4. sys.modules['foo'] = module (before execution!)
5. Execute foo.py code in module namespace
6. Bind name: caller.foo = module

Import variants:
  import foo          -> foo.bar()
  from foo import bar -> bar()
  import foo as f     -> f.bar()
  from foo import *   -> bar() (uses __all__)

Code

import sys

# sys.modules is the cache of all imported modules
print('os' in sys.modules)  # True (already imported by Python)
print('json' in sys.modules)  # May be False

import json
print('json' in sys.modules)  # True (now cached)

# Importing again returns the cached version
import json as j1
import json as j2
print(j1 is j2)  # True -- same object

# sys.path controls where Python looks for modules
print(sys.path[:3])  # ['', '/usr/lib/python3.x', ...]

# Import variants
import os                    # import module
from os.path import join     # import specific name
import os.path as osp        # import with alias

# Module attributes
print(json.__name__)    # 'json'
print(json.__file__)    # '/usr/lib/python3.x/json/__init__.py'
print(json.__package__) # 'json'

# Force reimport (for development/testing)
import importlib
importlib.reload(json)  # re-executes the module code

# Module is executed only once
# Create a test to demonstrate:
import types
test_module = types.ModuleType('test_mod')
exec('print("Module executed!"); VALUE = 42', test_module.__dict__)
sys.modules['test_mod'] = test_module

import test_mod  # does NOT print again -- cached
print(test_mod.VALUE)  # 42

# Cleanup
del sys.modules['test_mod']

# Import time side effects
# When you import, ALL top-level code in the module runs:
# - Function/class definitions (fast)
# - Variable assignments (fast)
# - Print statements, API calls, DB connections (slow, side effects!)
# Best practice: put side effects inside if __name__ == '__main__'

True
False
True
True
['', '/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12']
json
/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py
json
Module executed!
42

Key Points

•Modules are cached in sys.modules -- imported only once
•sys.path controls the search order for modules
•Module code is executed on first import (top-level statements run)
•importlib.reload() forces re-execution of module code

Gotchas

•Modifying a module after import doesn't require reimport (it's the same object)
•from module import x copies the reference -- reassigning x doesn't affect the module
•Circular imports work because sys.modules is populated before execution completes
•sys.path can be modified at runtime to add custom import locations

Python checks sys.modules cache first, then searches sys.path for the module file, executes it once, and caches it. Subsequent imports return the cached version.

•Modules cached in sys.modules, imported once
•sys.path controls search locations

import sys
import json

# Module is cached
print('json' in sys.modules)  # True

# Same object returned
import json as j
print(json is j)  # True

# Where Python looks
print(sys.path[:2])  # ['', '/usr/lib/...']

True
True
['', '/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python312.zip']

from module import x copies the reference at import time

Question 2 / 10

What is `init.py` and how do packages work in Python?

A Python package is a directory containing an __init__.py file. This file:

Marks the directory as a Python package
Is executed when the package is imported
Can define the package's public API
Can import submodules for convenience
Can define __all__ to control from package import *

Regular packages have __init__.py. Namespace packages (Python 3.3+) don't need __init__.py and can span multiple directories.

__init__.py can be empty (just marks the directory as a package) or contain initialization code, convenience imports, and API definitions.

Package structure example:

my_package/
    __init__.py
    module_a.py
    module_b.py
    sub_package/
        __init__.py
        module_c.py

Pseudo Code

Package structure:
  mypackage/
    __init__.py    -- package init, runs on import
    core.py        -- submodule
    utils.py       -- submodule
    sub/
      __init__.py  -- sub-package
      helpers.py   -- sub-submodule

__init__.py can:
  - Be empty (just marks as package)
  - Import submodules for convenience
  - Define __all__ for 'from package import *'
  - Run initialization code

Code

# Example __init__.py for a package
# mypackage/__init__.py

# Version info
__version__ = "1.0.0"
__author__ = "Alice"

# Control 'from mypackage import *'
__all__ = ["core", "utils", "Config"]

# Convenience imports -- so users can do:
# from mypackage import Config
# instead of: from mypackage.core import Config
# from .core import Config
# from .utils import helper_function

# Example package structure and usage:
# mypackage/
#   __init__.py
#   core.py        (contains Config class)
#   utils.py       (contains helper_function)
#   models/
#     __init__.py
#     user.py
#     product.py

# Users can import as:
# import mypackage
# from mypackage import Config
# from mypackage.models import user
# from mypackage.models.user import User

# Demonstrating with a real temporary package
import tempfile
import os
import sys

# Create a temporary package
pkg_dir = tempfile.mkdtemp()
pkg_path = os.path.join(pkg_dir, "demo_pkg")
os.makedirs(pkg_path)

# Write __init__.py
with open(os.path.join(pkg_path, "__init__.py"), "w") as f:
    f.write("""
__version__ = '1.0.0'
__all__ = ['greet']

def greet(name):
    return f'Hello, {name} from demo_pkg!'
""")

# Write a submodule
with open(os.path.join(pkg_path, "utils.py"), "w") as f:
    f.write("""
def add(a, b):
    return a + b
""")

# Add to path and import
sys.path.insert(0, pkg_dir)

import demo_pkg
print(demo_pkg.__version__)     # 1.0.0
print(demo_pkg.greet("World"))  # Hello, World from demo_pkg!

from demo_pkg.utils import add
print(add(1, 2))  # 3

# Cleanup
sys.path.remove(pkg_dir)
del sys.modules['demo_pkg']
del sys.modules['demo_pkg.utils']
import shutil
shutil.rmtree(pkg_dir)

1.0.0
Hello, World from demo_pkg!
3

Key Points

•__init__.py marks a directory as a package and runs on import
•Can be empty or contain convenience imports and __all__
•__all__ controls what 'from package import *' exports
•Namespace packages (3.3+) don't need __init__.py

Gotchas

•Heavy __init__.py slows down import (all top-level code runs)
•Circular imports in __init__.py are common -- use lazy imports or restructure
•from package import * only imports names in __all__ (if defined)
•Empty __init__.py is fine -- it just marks the directory as a package

__init__.py marks a directory as a Python package. It runs on import, can define the public API via __all__, and can provide convenience imports for users.

•__init__.py marks directory as package
•__all__ controls 'from package import *'

# mypackage/__init__.py
__version__ = "1.0.0"
__all__ = ["Config", "utils"]

# Convenience import
from .core import Config

# Usage:
# from mypackage import Config  # works due to __init__.py import

Heavy __init__.py slows import time

Question 3 / 10

How do relative and absolute imports work in Python?

Absolute imports use the full path from the project root:

from mypackage.utils import helper

Relative imports use dots to refer to the current and parent packages:

from . import utils          # current package
from .utils import helper     # current package's module
from .. import other_pkg      # parent package
from ..sibling import module  # sibling package

Dot notation:

. = current package
.. = parent package
... = grandparent package

PEP 8 recommends absolute imports for most cases because they're clearer. Relative imports are acceptable within a package to avoid hardcoding the package name.

Important: Relative imports only work inside packages (directories with __init__.py). They don't work in scripts run directly (python script.py).

Pseudo Code

Absolute:
  from mypackage.module import Class
  Explicit, clear, always works

Relative:
  from . import module        -- current package
  from .module import Class   -- current package module
  from .. import parent_mod   -- parent package
  from ..sibling import mod   -- sibling package

Rules:
  - Relative only works inside packages
  - Cannot use relative in __main__ script
  - PEP 8 prefers absolute imports

Code

# Package structure:
# myproject/
#   __init__.py
#   main.py
#   config.py
#   utils/
#     __init__.py
#     helpers.py
#     formatters.py
#   models/
#     __init__.py
#     user.py
#     product.py

# In myproject/utils/helpers.py:
# Absolute imports (preferred by PEP 8)
# from myproject.config import Settings
# from myproject.models.user import User

# Relative imports (acceptable within a package)
# from ..config import Settings         # parent package
# from ..models.user import User        # sibling package
# from .formatters import format_date   # same package
# from . import formatters              # same package module

# In myproject/models/user.py:
# from ..utils.helpers import validate  # parent's sibling
# from .product import Product          # same package

# Demonstration with actual code
import tempfile, os, sys

pkg = tempfile.mkdtemp()
base = os.path.join(pkg, "demo")
os.makedirs(os.path.join(base, "sub"))

# Create package files
with open(os.path.join(base, "__init__.py"), "w") as f:
    f.write("")

with open(os.path.join(base, "config.py"), "w") as f:
    f.write("DB_URL = 'sqlite:///app.db'\n")

with open(os.path.join(base, "sub", "__init__.py"), "w") as f:
    f.write("")

with open(os.path.join(base, "sub", "worker.py"), "w") as f:
    f.write("""
# Relative import from parent package
from ..config import DB_URL

def get_db():
    return f"Connecting to {DB_URL}"
""")

sys.path.insert(0, pkg)

from demo.sub.worker import get_db
print(get_db())  # Connecting to sqlite:///app.db

# Why relative imports fail in scripts:
# Running `python demo/sub/worker.py` directly fails because
# Python doesn't know the package context.
# Solution: run as module: python -m demo.sub.worker

# Cleanup
sys.path.remove(pkg)
for mod in list(sys.modules):
    if mod.startswith('demo'):
        del sys.modules[mod]
import shutil
shutil.rmtree(pkg)

Connecting to sqlite:///app.db

Key Points

•Absolute imports use full path from project root (PEP 8 preferred)
•Relative imports use dots: . = current, .. = parent package
•Relative imports only work inside packages, not in scripts run directly
•Use python -m package.module to run modules with relative imports

Gotchas

•Relative imports don't work when running a script directly (python script.py)
•from . import X fails in __main__ -- use python -m instead
•Mixing relative and absolute imports for the same package can cause confusion
•Relative imports can make refactoring (moving files) harder

Absolute imports use the full path (from pkg.mod import X). Relative imports use dots (. = current, .. = parent). PEP 8 prefers absolute. Relative only works inside packages.

•Absolute: full path, always clear
•Relative: dots for current/parent package

# Absolute (preferred)
from mypackage.utils import helper

# Relative (within package)
from . import utils            # current package
from .utils import helper      # current package module
from ..config import Settings  # parent package

Relative imports fail in directly-run scripts

Question 4 / 10

What does `all` do, and how does `from module import *` work?

__all__ is a list of strings that defines the public API of a module or package. It controls what is exported when someone uses from module import *.

If __all__ is defined: Only names listed in __all__ are imported with import *.

If __all__ is NOT defined: All names that don't start with underscore are imported with import *.

__all__ serves multiple purposes:

Controls import * behavior
Documents the module's public API
Helps IDEs and linters identify public symbols
In packages, controls which submodules import * exports

Note: __all__ does NOT prevent direct imports. from module import _private_func still works even if _private_func is not in __all__.

Pseudo Code

With __all__ defined:
  from module import *  -> imports only names in __all__

Without __all__:
  from module import *  -> imports all names not starting with _

__all__ does NOT:
  - Prevent direct imports (from mod import _private)
  - Affect regular imports (import mod; mod._private)
  - Enforce access control (it's just a convention)

Code

# Module with __all__ defined
# mymodule.py
"""
__all__ = ['public_func', 'PublicClass']

def public_func():
    return 'public'

def _private_func():
    return 'private'

def helper_func():
    return 'helper'

class PublicClass:
    pass

class _InternalClass:
    pass
"""

# from mymodule import *
# Only imports: public_func, PublicClass
# Does NOT import: _private_func, helper_func, _InternalClass

# Direct imports still work:
# from mymodule import helper_func  -- works!
# from mymodule import _private_func  -- works!

# Demonstration
import types
import sys

# Create module with __all__
mod = types.ModuleType('demo_mod')
mod.__all__ = ['greet', 'VERSION']
mod.greet = lambda name: f"Hello, {name}"
mod.VERSION = "1.0"
mod._internal = "secret"
mod.helper = "not exported"
sys.modules['demo_mod'] = mod

# Simulate 'from demo_mod import *'
# Python checks __all__ and only imports those names
namespace = {}
exec("from demo_mod import *", namespace)
print('greet' in namespace)     # True
print('VERSION' in namespace)   # True
print('_internal' in namespace) # False
print('helper' in namespace)    # False (not in __all__)

# Direct import still works
exec("from demo_mod import helper", namespace)
print(namespace['helper'])  # 'not exported'

# __all__ in packages (__init__.py)
# mypackage/__init__.py
# __all__ = ['module_a', 'module_b']  # controls submodule exports
# from mypackage import *  -> imports module_a and module_b

# Best practice: always define __all__ in public modules
# It documents the API and prevents accidental exports

# __all__ as documentation
def get_public_api(module):
    if hasattr(module, '__all__'):
        return module.__all__
    return [name for name in dir(module) if not name.startswith('_')]

import json
print(get_public_api(json)[:5])  # ['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', ...]

del sys.modules['demo_mod']

True
True
False
False
not exported
['dump', 'dumps', 'load', 'loads', 'JSONDecoder']

Key Points

•__all__ controls what 'from module import *' exports
•Without __all__, all non-underscore names are exported
•__all__ does NOT prevent direct imports
•Use __all__ to document and control the public API

Gotchas

•__all__ is a convention, not access control -- direct imports still work
•Forgetting to update __all__ when adding new public functions
•In packages, __all__ controls submodule export, not function export
•PEP 8 discourages 'from module import *' except in __init__.py

__all__ is a list of names exported by from module import *. Without it, all non-underscore names are exported. It documents the public API but doesn't prevent direct imports.

•__all__ controls import * behavior
•Does not prevent direct imports

# mymodule.py
__all__ = ['public_func', 'Config']

def public_func(): return "public"
def _private(): return "private"
def helper(): return "not exported by *"

# from mymodule import *  -> only public_func, Config
# from mymodule import helper  -> still works!

__all__ must be kept in sync with actual code

Question 5 / 10

What does `if name == 'main'` do, and why is it important?

__name__ is a special variable set by Python:

When a file is run directly: __name__ == '__main__'
When a file is imported: __name__ == 'module_name'

The if __name__ == '__main__': guard ensures that code only runs when the script is executed directly, not when it's imported as a module.

This is important because:

Prevents side effects on import: Test code, prints, and setup don't run when importing
Enables dual-use files: Same file works as both a module (importable) and a script (runnable)
Required for multiprocessing: Without the guard, child processes may re-execute code
Testing friendly: Can import functions without triggering script behavior

Pseudo Code

# script.py
def useful_function():
    return 42

# Code below only runs if script.py is run directly
if __name__ == '__main__':
    print(useful_function())

# Running directly:  python script.py -> prints 42
# Importing:         import script     -> nothing printed

Code

# Example: dual-use module/script
# calculator.py

def add(a, b):
    return a + b

def multiply(a, b):
    return a * b

def main():
    """Entry point when run as a script."""
    import sys
    if len(sys.argv) != 4:
        print("Usage: calculator.py <op> <a> <b>")
        sys.exit(1)

    op, a, b = sys.argv[1], float(sys.argv[2]), float(sys.argv[3])
    ops = {'add': add, 'mul': multiply}

    if op in ops:
        print(f"Result: {ops[op](a, b)}")
    else:
        print(f"Unknown operation: {op}")

if __name__ == '__main__':
    main()

# As a script: python calculator.py add 2 3 -> Result: 5.0
# As a module: from calculator import add; add(2, 3) -> 5

# __name__ values in different contexts
print(f"This module's __name__: {__name__}")
# If running directly: __main__
# If imported: the module name

import json
print(json.__name__)  # 'json'

import os
print(os.__name__)    # 'os'

# Package entry point: __main__.py
# mypackage/
#   __init__.py
#   __main__.py  <- runs with: python -m mypackage
#   core.py

# __main__.py allows:
# python -m mypackage  -> executes __main__.py

# Why it matters for multiprocessing
# Without the guard, each worker process re-imports the module
# and re-executes top-level code, causing infinite process spawning

# from multiprocessing import Pool
# def work(x):
#     return x * 2
# if __name__ == '__main__':  # REQUIRED for multiprocessing!
#     with Pool(4) as p:
#         print(p.map(work, range(10)))

Key Points

•__name__ is '__main__' when run directly, module name when imported
•Prevents code from running on import (side effects, tests, etc.)
•Enables files to work as both importable modules and runnable scripts
•Required for multiprocessing to prevent infinite process spawning

Gotchas

•Code above the guard runs on BOTH import and direct execution
•Heavy computation at module level (outside the guard) slows down imports
•python -m package runs __main__.py, not __init__.py
•In Jupyter notebooks, __name__ is always '__main__'

__name__ is '__main__' when run directly, module name when imported. The if __name__ == '__main__': guard prevents code from running on import.

•'__main__' when run directly, module name when imported
•Prevents side effects on import

def greet(name):
    return f"Hello, {name}!"

if __name__ == '__main__':
    # Only runs when script is executed directly
    print(greet("World"))  # not on import

# python script.py -> prints "Hello, World!"
# import script    -> nothing printed

Hello, World!

Code outside the guard always runs (on both import and direct execution)

Question 6 / 10

How do circular imports happen, and how do you fix them?

Circular imports occur when module A imports module B, and module B imports module A (directly or through a chain). This creates a dependency cycle.

Python handles circular imports by returning a partially initialized module from sys.modules. This means some names may not yet be defined when the import runs, causing ImportError or AttributeError.

The error typically manifests as ImportError: cannot import name 'X' from partially initialized module 'Y'.

Solutions:

Restructure: Move shared code to a third module
Import inside functions: Defer import to call time (lazy import)
Import the module, not the name: Use import module instead of from module import name
Use TYPE_CHECKING: For type hints that cause circular imports
Merge modules: If they're tightly coupled, they may belong together

Pseudo Code

Problem:
  a.py: from b import func_b
  b.py: from a import func_a
  -> ImportError on circular dependency

Fix 1: Restructure
  common.py: shared code
  a.py: from common import X
  b.py: from common import X

Fix 2: Lazy import
  def my_func():
      from b import func_b  # import at call time

Fix 3: Import module, not name
  import b  # instead of: from b import func_b
  b.func_b()  # access at call time

Code

# PROBLEM: Circular import
# a.py:
# from b import func_b
# def func_a(): return "A" + func_b()

# b.py:
# from a import func_a  # ImportError!
# def func_b(): return "B" + func_a()

# FIX 1: Move to function level (lazy import)
# a.py:
# def func_a():
#     from b import func_b  # deferred import
#     return "A" + func_b()

# FIX 2: Import module, not name
# a.py:
# import b
# def func_a():
#     return "A" + b.func_b()  # resolved at call time

# FIX 3: Restructure with a common module
# common.py:
# def shared_func(): return "shared"

# a.py:
# from common import shared_func
# def func_a(): return shared_func()

# FIX 4: TYPE_CHECKING for type hints
from __future__ import annotations  # makes all annotations strings
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    # Only imported during type checking (mypy), not at runtime
    from some_module import SomeClass

def process(obj: "SomeClass") -> None:  # string annotation
    pass

# Demonstration of the fix pattern
import types
import sys

# Create modules that would normally cause circular import
mod_a = types.ModuleType('mod_a')
mod_b = types.ModuleType('mod_b')

# Using lazy imports to break the cycle
mod_a_code = """
def func_a():
    import mod_b  # lazy import inside function
    return 'A calls ' + mod_b.func_b_safe()

def func_a_safe():
    return 'A'
"""

mod_b_code = """
def func_b():
    import mod_a  # lazy import inside function
    return 'B calls ' + mod_a.func_a_safe()

def func_b_safe():
    return 'B'
"""

sys.modules['mod_a'] = mod_a
sys.modules['mod_b'] = mod_b
exec(mod_a_code, mod_a.__dict__)
exec(mod_b_code, mod_b.__dict__)

print(mod_a.func_a())  # A calls B
print(mod_b.func_b())  # B calls A

# Cleanup
del sys.modules['mod_a']
del sys.modules['mod_b']

A calls B
B calls A

Key Points

•Circular imports happen when A imports B and B imports A
•Fix by restructuring, lazy imports, or importing modules instead of names
•TYPE_CHECKING guard for type-hint-only imports
•Circular imports are usually a sign of poor architecture

Gotchas

•from module import name fails with circular imports; import module may work
•Python returns partially initialized modules from sys.modules during circular imports
•Lazy imports add slight overhead and make dependencies less visible
•Circular imports usually indicate modules are too tightly coupled

Circular imports happen when A imports B and B imports A. Fix by restructuring, using lazy imports (inside functions), or importing modules instead of names.

•Restructure, lazy import, or import module not name
•TYPE_CHECKING for type-hint-only imports

# Problem: from b import X when b also imports from a

# Fix 1: Lazy import
def my_function():
    from other_module import helper  # import at call time
    return helper()

# Fix 2: TYPE_CHECKING for type hints
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from other_module import MyClass  # only for type checking

Usually a sign of poor architecture -- consider restructuring

Question 7 / 10

How does `sys.path` work, and how can you customize module search paths?

sys.path is a list of strings that specifies where Python looks for modules. When you import module, Python searches these directories in order.

sys.path is initialized from:

The directory of the script being run (or '' for interactive)
PYTHONPATH environment variable
Installation-dependent defaults (standard library, site-packages)

You can modify sys.path at runtime, but there are better approaches:

PYTHONPATH env var: Set before running Python
.pth files: Place in site-packages for persistent paths
pip install -e .: Editable installs for development
Package structure: Proper packaging with setup.py/pyproject.toml

Pseudo Code

sys.path search order:
1. Script directory (or '' for interactive)
2. PYTHONPATH directories
3. Standard library
4. site-packages (pip packages)

Modification options:
  sys.path.insert(0, '/my/path')  -- runtime
  PYTHONPATH=/my/path python ...  -- env var
  pip install -e .                -- development install

Code

import sys
import os

# Current sys.path
print("sys.path:")
for i, p in enumerate(sys.path[:5]):
    print(f"  {i}: {p}")

# First entry is the script's directory (or '' for interactive)
print(f"\nFirst entry: {sys.path[0]!r}")

# Adding a custom path at runtime
custom_path = "/tmp/my_modules"
os.makedirs(custom_path, exist_ok=True)

# Write a module to the custom path
with open(os.path.join(custom_path, "custom_mod.py"), "w") as f:
    f.write("CUSTOM_VALUE = 42\n")

# Add to sys.path
sys.path.insert(0, custom_path)  # insert at beginning for priority

import custom_mod
print(custom_mod.CUSTOM_VALUE)  # 42

# Remove from path
sys.path.remove(custom_path)

# site-packages location
import site
print(f"\nsite-packages: {site.getsitepackages()}")
print(f"user site-packages: {site.getusersitepackages()}")

# Finding where a module is located
import json
print(f"\njson location: {json.__file__}")

import os.path
print(f"os.path location: {os.path.__file__}")

# PYTHONPATH environment variable
# Set before running Python:
# export PYTHONPATH="/my/path:/other/path"
# python script.py

# .pth files for persistent paths
# Create a .pth file in site-packages:
# /path/to/site-packages/my_project.pth
# Contents: /path/to/my/project

# Best practice: use pip install -e for development
# In your project directory with setup.py or pyproject.toml:
# pip install -e .
# This adds your project to sys.path properly

# Cleanup
del sys.modules['custom_mod']
import shutil
shutil.rmtree(custom_path)

sys.path:
0:
1: /opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python312.zip
2: /opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12
3: /opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload
4: /Users/prakersh/Library/Python/3.12/lib/python/site-packages

First entry: ''
42

site-packages: ['/opt/homebrew/lib/python3.12/site-packages', '/opt/homebrew/opt/python@3.12/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages']
user site-packages: /Users/prakersh/Library/Python/3.12/lib/python/site-packages

json location: /opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py
os.path location: /opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12/posixpath.py

Key Points

•sys.path is searched in order for module files
•First entry is script directory, then PYTHONPATH, then stdlib, then site-packages
•sys.path.insert(0, path) for runtime customization
•pip install -e . is the proper way for development

Gotchas

•Modifying sys.path at runtime makes code fragile and non-portable
•Empty string '' in sys.path means current directory, which changes with os.chdir()
•PYTHONPATH persists for the entire session -- can cause unexpected imports
•sys.path modifications don't persist across Python sessions

sys.path is a list of directories Python searches for modules. It includes script dir, PYTHONPATH, stdlib, and site-packages. Use pip install -e . for development.

•sys.path searched in order: script dir, PYTHONPATH, stdlib, site-packages
•pip install -e . for development is the proper approach

import sys

# View current search paths
print(sys.path[:3])

# Add custom path (not recommended for production)
sys.path.insert(0, "/my/custom/path")

# Better: use PYTHONPATH env var or pip install -e .

['', '/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12']

Runtime sys.path modifications are fragile

Question 8 / 10

How does `importlib` enable dynamic imports and module reloading?

importlib is Python's programmatic interface to the import system. Key functions:

importlib.import_module(name): Import a module by string name (dynamic import)
importlib.reload(module): Re-execute a module's code (for development)
importlib.resources: Access data files in packages
importlib.metadata: Access package metadata (version, etc.)

Dynamic imports are useful for:

Plugin systems: Load modules based on configuration
Factory patterns: Instantiate classes by name
Conditional imports: Load platform-specific modules
Lazy loading: Import expensive modules only when needed

Pseudo Code

importlib.import_module('json')  -> same as 'import json'
importlib.import_module('.utils', 'mypackage')  -> relative import
importlib.reload(module)  -> re-execute module code

Use cases:
  - Plugin loading by name
  - Factory pattern with class names
  - Hot-reloading in development

Code

import importlib
import sys

# Dynamic import by string name
json_mod = importlib.import_module('json')
print(json_mod.dumps({"key": "value"}))  # {"key": "value"}

# Import submodule
os_path = importlib.import_module('os.path')
print(os_path.join('/tmp', 'file.txt'))  # /tmp/file.txt

# Plugin system pattern
def load_plugin(plugin_name):
    """Load a plugin module by name."""
    try:
        module = importlib.import_module(f"plugins.{plugin_name}")
        return module
    except ModuleNotFoundError:
        print(f"Plugin '{plugin_name}' not found")
        return None

# Factory pattern: create class by string name
def create_object(module_path, class_name, *args, **kwargs):
    """Create an object from module path and class name."""
    module = importlib.import_module(module_path)
    cls = getattr(module, class_name)
    return cls(*args, **kwargs)

# Example: create a datetime object dynamically
from datetime import datetime
dt = create_object('datetime', 'datetime', 2024, 1, 15)
print(dt)  # 2024-01-15 00:00:00

# Module reloading (development use)
import types
mod = types.ModuleType('dev_mod')
mod.__file__ = '<dynamic>'
exec('VERSION = 1', mod.__dict__)
sys.modules['dev_mod'] = mod

import dev_mod
print(dev_mod.VERSION)  # 1

# Simulate code change
exec('VERSION = 2', mod.__dict__)
importlib.reload(dev_mod)
print(dev_mod.VERSION)  # 2

# Conditional/platform-specific imports
def get_platform_module():
    import platform
    os_name = platform.system().lower()
    try:
        return importlib.import_module(f"platform_support.{os_name}")
    except ModuleNotFoundError:
        return importlib.import_module("platform_support.generic")

# Lazy import pattern
class LazyModule:
    def __init__(self, name):
        self._name = name
        self._module = None

    def __getattr__(self, attr):
        if self._module is None:
            self._module = importlib.import_module(self._name)
        return getattr(self._module, attr)

# numpy is only imported when first used
# np = LazyModule('numpy')
# np.array([1, 2, 3])  # imports numpy here

# Package metadata (Python 3.8+)
from importlib.metadata import version, packages_distributions
try:
    print(f"pip version: {version('pip')}")
except Exception:
    pass

del sys.modules['dev_mod']

{"key": "value"}
/tmp/file.txt
2024-01-15 00:00:00
1
2
pip version: 24.0  (varies by installation)

Key Points

•import_module() imports by string name (dynamic/runtime import)
•reload() re-executes module code for development
•Enables plugin systems, factory patterns, and lazy loading
•importlib.metadata provides package version info

Gotchas

•reload() doesn't update existing references to old objects
•reload() may leave stale state if module has initialization side effects
•Dynamic imports make code harder to analyze statically (IDE support, type checkers)
•import_module with relative names requires the package parameter

importlib.import_module('name') imports dynamically by string. importlib.reload(mod) re-executes module code. Used for plugins, factories, and lazy loading.

•import_module for dynamic imports by string
•reload for development hot-reloading

import importlib

# Dynamic import by string
json_mod = importlib.import_module('json')
print(json_mod.dumps({"a": 1}))  # {"a": 1}

# Reload module (development)
import my_module
importlib.reload(my_module)  # re-execute module code

reload doesn't update existing object references

Question 9 / 10

What are namespace packages, and how do they differ from regular packages?

Regular packages have an __init__.py file. Namespace packages (PEP 420, Python 3.3+) don't require __init__.py and can span multiple directories.

Namespace packages allow different parts of a package to be distributed separately and installed in different locations. Python finds all directories matching the package name across sys.path and merges them into a single namespace.

Use cases:

Large frameworks distributed as separate installable components
Plugin architectures where plugins add subpackages
Organization-wide shared package namespaces (e.g., google.cloud.storage)

Key differences:

No __init__.py file
No __file__ attribute (multiple directories)
__path__ contains multiple directories
Cannot have code at the namespace package level (no __init__.py to put it in)

Pseudo Code

Regular package:
  mypackage/
    __init__.py  <- required
    module.py

Namespace package:
  location1/mypackage/module_a.py
  location2/mypackage/module_b.py
  No __init__.py anywhere!
  Both locations in sys.path
  -> import mypackage.module_a  (from location1)
  -> import mypackage.module_b  (from location2)

Code

import sys
import os
import tempfile

# Create two separate directories, each with part of the same package
dir1 = tempfile.mkdtemp()
dir2 = tempfile.mkdtemp()

# Create namespace package parts (NO __init__.py!)
ns_dir1 = os.path.join(dir1, "my_namespace")
ns_dir2 = os.path.join(dir2, "my_namespace")
os.makedirs(ns_dir1)
os.makedirs(ns_dir2)

# Part 1: module_a in dir1
with open(os.path.join(ns_dir1, "module_a.py"), "w") as f:
    f.write("def func_a(): return 'from module_a'\n")

# Part 2: module_b in dir2
with open(os.path.join(ns_dir2, "module_b.py"), "w") as f:
    f.write("def func_b(): return 'from module_b'\n")

# Add both directories to sys.path
sys.path.insert(0, dir1)
sys.path.insert(0, dir2)

# Import from both parts of the namespace package
from my_namespace.module_a import func_a
from my_namespace.module_b import func_b

print(func_a())  # from module_a
print(func_b())  # from module_b

# Inspect the namespace package
import my_namespace
print(type(my_namespace))    # <class 'module'>
print(hasattr(my_namespace, '__file__'))  # False (no single file)
print(my_namespace.__path__)  # _NamespacePath([dir2/my_namespace, dir1/my_namespace])

# Compare with regular package
# Regular package has __file__ pointing to __init__.py
import json
print(hasattr(json, '__file__'))  # True
print(json.__file__)  # .../json/__init__.py

# Real-world example: google-cloud packages
# pip install google-cloud-storage google-cloud-bigquery
# Both install into google/cloud/ namespace
# from google.cloud import storage
# from google.cloud import bigquery
# No google/__init__.py or google/cloud/__init__.py!

# Cleanup
sys.path.remove(dir1)
sys.path.remove(dir2)
for mod in list(sys.modules):
    if 'my_namespace' in mod:
        del sys.modules[mod]
import shutil
shutil.rmtree(dir1)
shutil.rmtree(dir2)

from module_a
from module_b
<class 'module'>
True
_NamespacePath(['/var/folders/qz/99sz9qnd1gld4tqq0rgyd41m0000gn/T/tmplnmjjuq2/my_namespace', '/var/folders/qz/99sz9qnd1gld4tqq0rgyd41m0000gn/T/tmpatbeolpb/my_namespace'])
True
/opt/homebrew/Cellar/python@3.12/3.12.13/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py

Key Points

•Namespace packages have no __init__.py and can span multiple directories
•Python merges all matching directories into one namespace
•__path__ contains multiple directories instead of one
•Used for distributing package parts independently (google-cloud, etc.)

Gotchas

•Adding __init__.py to a namespace package directory converts it to regular (breaks merging)
•Namespace packages are slower to import (must search all sys.path entries)
•Cannot have package-level code (no __init__.py to put it in)
•Debugging namespace packages is harder (code in multiple locations)

Namespace packages have no __init__.py and can span multiple directories. Python merges all matching directories into one namespace. Used for separately distributed package parts.

•No __init__.py needed
•Merges multiple directories into one namespace

# Two directories, no __init__.py:
# /path1/mypkg/module_a.py
# /path2/mypkg/module_b.py

# Both in sys.path:
import sys
sys.path.extend(["/path1", "/path2"])

from mypkg.module_a import func_a  # from /path1
from mypkg.module_b import func_b  # from /path2

Adding __init__.py breaks the namespace merging

Question 10 / 10

What are the best practices for structuring Python packages?

A well-structured Python package follows conventions that make it easy to install, test, and maintain.

Recommended project layout (src layout):

my_project/
    pyproject.toml
    README.md
    src/
        my_package/
            __init__.py
            core.py
            utils.py
            models/
                __init__.py
                user.py
    tests/
        __init__.py
        test_core.py
        test_utils.py

Key principles:

Use pyproject.toml for project metadata and build config
Use src/ layout to prevent accidental imports from the project root
Keep __init__.py minimal (imports and __all__)
Separate tests from source code
One class/concept per module (don't overstuff)
Use absolute imports for clarity
Define __all__ in public modules

Pseudo Code

project/
  pyproject.toml       -- project metadata, dependencies
  src/
    mypackage/
      __init__.py      -- minimal: version, __all__, convenience imports
      core.py          -- main functionality
      utils.py         -- helpers
      exceptions.py    -- custom exceptions
      models/
        __init__.py
        user.py
  tests/
    test_core.py
    test_utils.py
    conftest.py        -- pytest fixtures

Code

# pyproject.toml (modern packaging)
# [project]
# name = "my-package"
# version = "1.0.0"
# requires-python = ">=3.8"
# dependencies = ["requests>=2.28"]
#
# [project.optional-dependencies]
# dev = ["pytest", "mypy", "ruff"]
#
# [build-system]
# requires = ["setuptools>=64"]
# build-backend = "setuptools.backends._legacy:_Backend"

# src/mypackage/__init__.py (minimal and clean)
# __version__ = "1.0.0"
# __all__ = ["Client", "Config", "AppError"]
#
# from .core import Client
# from .config import Config
# from .exceptions import AppError

# src/mypackage/exceptions.py (custom exceptions)
# class AppError(Exception):
#     pass
# class NotFoundError(AppError):
#     pass
# class ValidationError(AppError):
#     pass

# src/mypackage/core.py (main module)
# from .config import Config
# from .exceptions import NotFoundError
#
# class Client:
#     def __init__(self, config: Config):
#         self.config = config

# Installation and usage:
# pip install -e .            # development install
# pip install -e ".[dev]"     # with dev dependencies
# from mypackage import Client, Config

# Testing structure:
# tests/conftest.py
# import pytest
# from mypackage import Client, Config
#
# @pytest.fixture
# def client():
#     return Client(Config(debug=True))

# tests/test_core.py
# def test_client_creation(client):
#     assert client is not None

# Why src/ layout is better:
# Without src/:
#   my_project/
#     my_package/
#     tests/
#   Running tests from my_project/ imports local my_package/
#   even if it's not installed -- hides import bugs
#
# With src/:
#   my_project/
#     src/my_package/
#     tests/
#   Must install (pip install -e .) to import
#   Tests always use the installed version

# Module organization principles
print("""
Package Structure Best Practices:
1. pyproject.toml for metadata (not setup.py)
2. src/ layout to prevent accidental imports
3. Minimal __init__.py with __all__
4. One responsibility per module
5. Custom exceptions in exceptions.py
6. Tests mirror source structure
7. conftest.py for shared fixtures
8. Use absolute imports in source
9. Type hints on all public functions
10. Docstrings on all public modules/classes/functions
""")


Package Structure Best Practices:
1. pyproject.toml for metadata (not setup.py)
2. src/ layout to prevent accidental imports
3. Minimal __init__.py with __all__
4. One responsibility per module
5. Custom exceptions in exceptions.py
6. Tests mirror source structure
7. conftest.py for shared fixtures
8. Use absolute imports in source
9. Type hints on all public functions
10. Docstrings on all public modules/classes/functions

Key Points

•Use src/ layout to prevent accidental imports
•pyproject.toml for project metadata (modern standard)
•Minimal __init__.py with __all__ and convenience imports
•Separate tests from source, mirror the package structure

Gotchas

•Without src/ layout, tests may accidentally import uninstalled local code
•Overstuffing __init__.py with code slows down all imports of the package
•Forgetting __init__.py in test directories can cause pytest discovery issues
•Circular imports in __init__.py are common -- use lazy imports if needed

Use src/ layout, pyproject.toml, minimal __init__.py with __all__, and separate tests/ directory. Install with pip install -e . for development.

•src/ layout, pyproject.toml, minimal __init__.py
•pip install -e . for development

# Project structure:
# my_project/
#   pyproject.toml
#   src/
#     mypackage/
#       __init__.py   -> __all__ = ['Client']
#       core.py       -> class Client: ...
#   tests/
#     test_core.py

# Install for development:
# pip install -e .

# Then import normally:
# from mypackage import Client

Without src/, tests may import uninstalled local code

Built by onllm.dev

Home GitHub Support awesomePrep

How does Python's import system work, and what happens when you import a module?

What is `__init__.py` and how do packages work in Python?

How do relative and absolute imports work in Python?

What does `__all__` do, and how does `from module import *` work?

What does `if __name__ == '__main__'` do, and why is it important?

How do circular imports happen, and how do you fix them?

How does `sys.path` work, and how can you customize module search paths?

How does `importlib` enable dynamic imports and module reloading?

What are namespace packages, and how do they differ from regular packages?

What are the best practices for structuring Python packages?

Modules, Packages & Imports

What is `init.py` and how do packages work in Python?

What does `all` do, and how does `from module import *` work?

What does `if name == 'main'` do, and why is it important?