← Python

Python Internals & Performance

← Python Internals & Performance

Select a question from the sidebar

15 questions in this topic

Question 1 / 15

What is CPython and how does the Python compilation process work?

CPython is the reference implementation of Python, written in C. When you type python, you are almost always running CPython. Other implementations include PyPy (JIT-compiled), Jython (JVM), and IronPython (.NET).

The compilation process:

Source code (.py) is read by the parser.
Abstract Syntax Tree (AST) is generated from the parsed source.
Bytecode (.pyc) is compiled from the AST. This is an intermediate representation, not machine code.
Python Virtual Machine (PVM) executes the bytecode instruction by instruction.

Bytecode is cached in __pycache__/ directories as .pyc files. Python recompiles only when the source file is newer than the cached bytecode.

The PVM is a stack-based virtual machine: operations push and pop values on an evaluation stack.

Pseudo Code

1. source.py -> Lexer/Parser -> AST
2. AST -> Compiler -> Bytecode (.pyc)
3. Bytecode -> PVM (stack-based VM) -> Execution
4. Bytecode is cached in __pycache__/
5. Recompiled only when .py is newer than .pyc

Code

import dis
import ast
import sys

# 1. View the AST
source = "x = 1 + 2"
tree = ast.parse(source)
print(ast.dump(tree, indent=2))
# Module(body=[
#   Assign(targets=[Name(id='x')],
#          value=BinOp(left=Constant(value=1), op=Add(), right=Constant(value=2)))])

# 2. View bytecode
def add(a, b):
    return a + b

dis.dis(add)
#   2           0 LOAD_FAST                0 (a)
#               2 LOAD_FAST                1 (b)
#               4 BINARY_ADD
#               6 RETURN_VALUE

# 3. Code object internals
code = add.__code__
print(f"Name: {code.co_name}")         # add
print(f"Arg count: {code.co_argcount}") # 2
print(f"Constants: {code.co_consts}")   # (None,)
print(f"Local vars: {code.co_varnames}") # ('a', 'b')
print(f"Bytecode: {code.co_code}")      # Raw bytes
print(f"Stack size: {code.co_stacksize}") # Max stack depth

# 4. Compile manually
code_obj = compile('print("hello")', '<string>', 'exec')
exec(code_obj)  # hello

# 5. Check Python implementation
print(sys.implementation.name)  # cpython
print(sys.version)              # 3.x.x

# 6. CPython constant folding optimization
def optimized():
    x = 2 * 3 * 4  # Compiled as x = 24 (computed at compile time)
    return x

dis.dis(optimized)
# LOAD_CONST   1 (24)   -- Not 2, 3, 4 separately!

Module(
body=[
Assign(
targets=[
Name(id='x', ctx=Store())],
value=BinOp(
left=Constant(value=1),
op=Add(),
right=Constant(value=2)))],
type_ignores=[])
14           0 RESUME                   0

15           2 LOAD_FAST                0 (a)
4 LOAD_FAST                1 (b)
6 BINARY_OP                0 (+)
10 RETURN_VALUE
Name: add
Arg count: 2
Constants: (None,)
Local vars: ('a', 'b')
Bytecode: b'\x97\x00|\x00|\x01z\x00\x00\x00S\x00'
Stack size: 2
hello
cpython
3.12.13 (main, Mar  3 2026, 12:39:30) [Clang 17.0.0 (clang-1700.6.3.2)]
41           0 RESUME                   0

42           2 LOAD_CONST               1 (24)
4 STORE_FAST               0 (x)

43           6 LOAD_FAST                0 (x)
8 RETURN_VALUE

Key Points

•CPython compiles .py to bytecode (.pyc), then the PVM interprets the bytecode
•dis module disassembles bytecode for inspection
•ast module shows the Abstract Syntax Tree before compilation
•Bytecode is cached in __pycache__/ as .pyc files for faster loading
•CPython performs optimizations like constant folding at compile time

Gotchas

•Python is compiled (to bytecode), then interpreted; it is not purely interpreted
•Bytecode is NOT machine code; it runs on the PVM, not directly on the CPU
•.pyc files are implementation-specific; CPython's bytecodes differ from PyPy's

CPython compiles Python source to bytecode (.pyc), which runs on the Python Virtual Machine (PVM). View bytecode with dis.dis(), AST with ast.parse(). Bytecode is cached in __pycache__/.

•Source -> AST -> bytecode -> PVM execution
•dis.dis() shows bytecode; cached in __pycache__/

import dis

def add(a, b):
    return a + b

dis.dis(add)
#  LOAD_FAST  0 (a)
#  LOAD_FAST  1 (b)
#  BINARY_ADD
#  RETURN_VALUE

Python is compiled to bytecode, not directly interpreted from source

Question 2 / 15

What is the GIL (Global Interpreter Lock) and how does it affect Python performance?

The GIL (Global Interpreter Lock) is a mutex in CPython that allows only one thread to execute Python bytecode at a time, even on multi-core machines.

Why does the GIL exist? CPython uses reference counting for memory management. The GIL prevents race conditions on the reference count, which would cause memory leaks or use-after-free bugs without it.

Impact:

CPU-bound tasks: Threads do NOT run in parallel. A CPU-bound multi-threaded program runs at single-core speed.
I/O-bound tasks: The GIL is released during I/O operations (network, disk, sleep), so threading works well.
C extensions: Well-written C extensions (NumPy, etc.) release the GIL, enabling true parallelism.

Workarounds:

multiprocessing: Each process has its own Python interpreter and GIL.
asyncio: Cooperative multitasking for I/O-bound work.
C extensions: NumPy, Cython release the GIL for numerical work.
Python 3.13+: Experimental free-threaded build (no-GIL).

Pseudo Code

GIL = mutex on Python bytecode execution
Only 1 thread runs Python at a time
Released during I/O (network, disk)
CPU-bound: use multiprocessing
I/O-bound: threading and asyncio work fine

Code

import threading
import multiprocessing
import time

def cpu_bound(n):
    """CPU-intensive: count to n."""
    total = 0
    for i in range(n):
        total += i
    return total

def io_bound(seconds):
    """I/O-bound: sleep."""
    time.sleep(seconds)

# --- CPU-bound: GIL prevents parallel execution ---
N = 10_000_000

# Single-threaded
start = time.time()
cpu_bound(N)
cpu_bound(N)
print(f"Sequential: {time.time() - start:.2f}s")

# Multi-threaded (NOT faster due to GIL)
start = time.time()
t1 = threading.Thread(target=cpu_bound, args=(N,))
t2 = threading.Thread(target=cpu_bound, args=(N,))
t1.start(); t2.start()
t1.join(); t2.join()
print(f"Threaded: {time.time() - start:.2f}s")  # Same or slower!

# Multi-process (truly parallel, bypasses GIL)
start = time.time()
p1 = multiprocessing.Process(target=cpu_bound, args=(N,))
p2 = multiprocessing.Process(target=cpu_bound, args=(N,))
p1.start(); p2.start()
p1.join(); p2.join()
print(f"Multiprocess: {time.time() - start:.2f}s")  # ~2x faster!

# --- I/O-bound: GIL is released, threading works ---
start = time.time()
io_bound(1)
io_bound(1)
print(f"Sequential I/O: {time.time() - start:.2f}s")  # ~2s

start = time.time()
t1 = threading.Thread(target=io_bound, args=(1,))
t2 = threading.Thread(target=io_bound, args=(1,))
t1.start(); t2.start()
t1.join(); t2.join()
print(f"Threaded I/O: {time.time() - start:.2f}s")  # ~1s (parallel!)

# Check GIL status
import sys
print(sys.version_info)  # Check Python version
# Python 3.13+ with --disable-gil flag for free-threaded mode

# Expected Output: (timing values are approximate)
# Sequential: 0.41s
# Threaded: 0.38s
# Multiprocess: 0.04s
# Sequential I/O: 2.01s
# Threaded I/O: 1.00s
# sys.version_info(major=3, minor=12, micro=13, releaselevel='final', serial=0)

Key Points

•GIL allows only one thread to execute Python bytecode at a time
•Exists to protect CPython's reference counting memory management
•Threading helps I/O-bound tasks (GIL released during I/O) but not CPU-bound
•Use multiprocessing for CPU-bound parallelism (separate GILs per process)
•Python 3.13 introduces experimental free-threaded (no-GIL) mode

Gotchas

•The GIL does not prevent all race conditions; you still need locks for shared data structures
•NumPy, scikit-learn, and other C extensions release the GIL and achieve true parallelism
•multiprocessing has higher overhead (process creation, data serialization) than threading

The GIL lets only one thread run Python bytecode at a time. Threading helps I/O-bound tasks (GIL released during I/O) but not CPU-bound. Use multiprocessing for CPU-bound parallelism. Python 3.13+ has experimental no-GIL mode.

•GIL: 1 thread runs Python at a time; released during I/O
•multiprocessing for CPU-bound; threading for I/O-bound

# CPU-bound: use multiprocessing (bypasses GIL)
from multiprocessing import Pool
with Pool(4) as pool:
    results = pool.map(cpu_task, data_chunks)

# I/O-bound: threading works fine (GIL released during I/O)
import threading
t = threading.Thread(target=fetch_url, args=(url,))
t.start()

GIL does not prevent race conditions on shared data

Question 3 / 15

What are slots and how do they optimize memory usage?

By default, Python objects store their attributes in a __dict__ dictionary. This is flexible (you can add any attribute at runtime) but uses more memory.

__slots__ replaces __dict__ with a fixed set of attribute slots, implemented as C-level descriptors. Benefits:

Less memory: No per-instance __dict__ (saves ~40-60 bytes per instance).
Faster attribute access: Slots use fixed offsets instead of dict hash lookups.
Prevents typos: Cannot add attributes not declared in __slots__.

Trade-offs: No dynamic attributes, no __dict__, more rigid.

Pseudo Code

1. Define __slots__ = ('attr1', 'attr2') in the class
2. Python allocates fixed slots instead of __dict__
3. Cannot add attributes not in __slots__
4. Saves ~40-60 bytes per instance
5. Slightly faster attribute access

Code

import sys

# Without __slots__ (default)
class PointDict:
    def __init__(self, x, y):
        self.x = x
        self.y = y

# With __slots__
class PointSlots:
    __slots__ = ('x', 'y')

    def __init__(self, x, y):
        self.x = x
        self.y = y

# Memory comparison
p_dict = PointDict(1, 2)
p_slots = PointSlots(1, 2)

print(sys.getsizeof(p_dict))           # ~48 bytes (object)
print(sys.getsizeof(p_dict.__dict__))  # ~104 bytes (dict)
# Total: ~152 bytes

print(sys.getsizeof(p_slots))  # ~56 bytes (no dict!)

# Dynamic attribute test
p_dict.z = 3     # OK: __dict__ allows dynamic attributes
# p_slots.z = 3  # AttributeError: 'PointSlots' has no attribute 'z'

# __slots__ prevents __dict__
print(hasattr(p_dict, '__dict__'))   # True
print(hasattr(p_slots, '__dict__'))  # False

# __slots__ with inheritance
class Point3D(PointSlots):
    __slots__ = ('z',)  # Only NEW attributes; x, y inherited

    def __init__(self, x, y, z):
        super().__init__(x, y)
        self.z = z

p3d = Point3D(1, 2, 3)
print(p3d.x, p3d.y, p3d.z)  # 1 2 3

# __slots__ with default values
class Config:
    __slots__ = ('debug', 'host', 'port')

    def __init__(self, host='localhost', port=8080, debug=False):
        self.host = host
        self.port = port
        self.debug = debug

# Memory savings at scale
from dataclasses import dataclass

@dataclass
class UserDict:
    name: str
    age: int
    email: str

@dataclass
class UserSlots:
    __slots__ = ('name', 'age', 'email')
    name: str
    age: int
    email: str

# Create 100,000 instances
users_dict = [UserDict('Alice', 30, 'a@b.com') for _ in range(100_000)]
users_slots = [UserSlots('Alice', 30, 'a@b.com') for _ in range(100_000)]

# Slots saves significant memory at scale
# dataclasses(slots=True) in Python 3.10+
@dataclass(slots=True)
class UserSlotsModern:
    name: str
    age: int
    email: str

48
296
48
True
False
1 2 3

Key Points

•__slots__ replaces __dict__ with fixed C-level attribute slots
•Saves ~40-60 bytes per instance; significant for millions of objects
•Prevents adding undeclared attributes (catches typos)
•Slightly faster attribute access due to fixed offsets
•Python 3.10 dataclass(slots=True) is the modern way to use slots

Gotchas

•Classes with __slots__ cannot have __dict__ (no dynamic attributes) unless you add '__dict__' to slots
•Subclasses must define their own __slots__ for new attributes; repeating parent slots wastes memory
•__slots__ does not work with multiple inheritance if both parents have non-empty slots

__slots__ replaces the per-instance __dict__ with fixed attribute slots, saving ~40-60 bytes per object. Use for memory-critical classes with many instances. Python 3.10+ supports @dataclass(slots=True).

•Fixed slots instead of __dict__; saves memory at scale
•dataclass(slots=True) is the modern approach

class Point:
    __slots__ = ('x', 'y')
    def __init__(self, x, y):
        self.x = x
        self.y = y

# p.z = 3  # AttributeError: can't add attributes

# Modern approach
from dataclasses import dataclass

@dataclass(slots=True)
class Point:
    x: float
    y: float

Cannot add dynamic attributes; subclasses need their own __slots__

Question 4 / 15

What are descriptors in Python and how do get, set, and delete work?

Descriptors are objects that define __get__, __set__, or __delete__ methods. When placed as a class attribute, they intercept attribute access on instances of that class.

Descriptors power many Python features: property, classmethod, staticmethod, __slots__, and super() all use the descriptor protocol.

Types:

Data descriptor: Defines __set__ and/or __delete__ (takes priority over instance __dict__).
Non-data descriptor: Defines only __get__ (instance __dict__ takes priority).

Attribute lookup order: data descriptor > instance __dict__ > non-data descriptor > class __dict__ > __getattr__.

Pseudo Code

1. Define a class with __get__(self, obj, objtype)
2. __set__(self, obj, value) for setting
3. __delete__(self, obj) for deleting
4. Place the descriptor as a CLASS attribute
5. Access on an instance triggers the descriptor methods

Code

# Basic descriptor
class Validated:
    """Descriptor that validates values."""

    def __init__(self, name, validator):
        self.name = name
        self.validator = validator

    def __set_name__(self, owner, name):
        """Called when the descriptor is assigned to a class attribute."""
        self.name = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self  # Access from class returns descriptor itself
        return obj.__dict__.get(self.name)

    def __set__(self, obj, value):
        if not self.validator(value):
            raise ValueError(f"Invalid value for {self.name}: {value}")
        obj.__dict__[self.name] = value

    def __delete__(self, obj):
        del obj.__dict__[self.name]

# Usage
class User:
    name = Validated('name', lambda v: isinstance(v, str) and len(v) > 0)
    age = Validated('age', lambda v: isinstance(v, int) and 0 < v < 150)
    email = Validated('email', lambda v: isinstance(v, str) and '@' in v)

user = User()
user.name = 'Alice'      # Calls Validated.__set__
user.age = 30            # Validated
user.email = 'a@b.com'   # Validated

print(user.name)  # Calls Validated.__get__ -> 'Alice'

# user.age = -1    # ValueError: Invalid value for age: -1
# user.email = 'no-at'  # ValueError

# Type-checking descriptor
class TypedField:
    def __init__(self, expected_type):
        self.expected_type = expected_type

    def __set_name__(self, owner, name):
        self.name = name

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        return obj.__dict__.get(self.name)

    def __set__(self, obj, value):
        if not isinstance(value, self.expected_type):
            raise TypeError(
                f"{self.name} must be {self.expected_type.__name__}, "
                f"got {type(value).__name__}"
            )
        obj.__dict__[self.name] = value

class Product:
    name = TypedField(str)
    price = TypedField(float)
    quantity = TypedField(int)

    def __init__(self, name, price, quantity):
        self.name = name
        self.price = price
        self.quantity = quantity

p = Product('Widget', 9.99, 100)
# p.price = 'free'  # TypeError: price must be float, got str

# How property() is a descriptor under the hood
class Property:
    """Simplified reimplementation of built-in property."""

    def __init__(self, fget=None, fset=None, fdel=None):
        self.fget = fget
        self.fset = fset
        self.fdel = fdel

    def __get__(self, obj, objtype=None):
        if obj is None:
            return self
        if self.fget is None:
            raise AttributeError('unreadable attribute')
        return self.fget(obj)

    def __set__(self, obj, value):
        if self.fset is None:
            raise AttributeError("can't set attribute")
        self.fset(obj, value)

    def setter(self, fset):
        return type(self)(self.fget, fset, self.fdel)

Alice

Key Points

•Descriptors intercept attribute access via __get__, __set__, __delete__
•Data descriptors (with __set__) take priority over instance __dict__
•Non-data descriptors (only __get__) are overridden by instance __dict__
•__set_name__ is called when the descriptor is assigned to a class attribute
•property, classmethod, staticmethod are all implemented as descriptors

Gotchas

•Descriptors must be class attributes, not instance attributes
•Data descriptors always win over instance __dict__; non-data descriptors lose to it
•Store per-instance data in obj.__dict__, not on the descriptor itself (shared across instances)

Descriptors define __get__, __set__, __delete__ to intercept attribute access. Placed as class attributes, they control how instances access those attributes. property, classmethod, staticmethod are all descriptors.

•__get__/__set__/__delete__ intercept attribute access on instances
•Data descriptors override instance __dict__; property is a descriptor

class TypedField:
    def __init__(self, expected_type):
        self.type = expected_type
    def __set_name__(self, owner, name):
        self.name = name
    def __get__(self, obj, cls):
        return obj.__dict__.get(self.name) if obj else self
    def __set__(self, obj, value):
        if not isinstance(value, self.type):
            raise TypeError(f"{self.name} must be {self.type.__name__}")
        obj.__dict__[self.name] = value

class User:
    name = TypedField(str)
    age = TypedField(int)

Store per-instance data in obj.__dict__, not on the descriptor

Question 5 / 15

What are metaclasses in Python and how does type() work?

In Python, classes are objects too. Just as objects are instances of classes, classes are instances of metaclasses. The default metaclass is type.

type has a dual role:

type(obj) returns the type of an object.
type(name, bases, dict) creates a new class dynamically.

A metaclass is a class whose instances are classes. By defining a custom metaclass, you can control class creation: add methods, validate class definitions, register classes, etc.

The class creation chain: type.__call__ calls type.__new__ (creates the class) then type.__init__ (initializes it).

Pseudo Code

1. type is the metaclass of all classes (by default)
2. type(name, bases, dict) creates a class dynamically
3. Custom metaclass: class Meta(type): def __new__(cls, name, bases, dict)
4. class MyClass(metaclass=Meta): triggers Meta.__new__
5. __init_subclass__ is a simpler alternative for many use cases

Code

# type() creates classes dynamically
MyClass = type('MyClass', (object,), {
    'x': 42,
    'greet': lambda self: f"Hello from {type(self).__name__}"
})

obj = MyClass()
print(obj.x)       # 42
print(obj.greet())  # Hello from MyClass

# Everything is an instance of type
print(type(42))           # <class 'int'>
print(type(int))          # <class 'type'>
print(type(type))         # <class 'type'> (type is its own metaclass!)

# Custom metaclass
class RegistryMeta(type):
    """Metaclass that auto-registers all classes."""
    _registry = {}

    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        if bases:  # Don't register the base class itself
            RegistryMeta._registry[name] = cls
        return cls

    @classmethod
    def get_registry(mcs):
        return dict(mcs._registry)

class Plugin(metaclass=RegistryMeta):
    pass

class AudioPlugin(Plugin):
    def process(self):
        return "Processing audio"

class VideoPlugin(Plugin):
    def process(self):
        return "Processing video"

# Plugins are auto-registered
print(RegistryMeta.get_registry())
# {'AudioPlugin': <class 'AudioPlugin'>, 'VideoPlugin': <class 'VideoPlugin'>}

# Metaclass for validation
class ValidatedMeta(type):
    def __new__(mcs, name, bases, namespace):
        # Ensure all subclasses implement required methods
        if bases:  # Skip the base class
            required = getattr(bases[0], '_required_methods', [])
            for method in required:
                if method not in namespace:
                    raise TypeError(
                        f"{name} must implement {method}()"
                    )
        return super().__new__(mcs, name, bases, namespace)

class BaseHandler(metaclass=ValidatedMeta):
    _required_methods = ['handle', 'validate']

# class BadHandler(BaseHandler):  # TypeError: must implement handle()
#     pass

class GoodHandler(BaseHandler):
    def handle(self): pass
    def validate(self): pass

# Simpler alternative: __init_subclass__ (Python 3.6+)
class PluginBase:
    _registry = {}

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        PluginBase._registry[cls.__name__] = cls

class MyPlugin(PluginBase):
    pass

print(PluginBase._registry)  # {'MyPlugin': <class 'MyPlugin'>}

42
Hello from MyClass
<class 'int'>
<class 'type'>
<class 'type'>
{'AudioPlugin': <class '__main__.AudioPlugin'>, 'VideoPlugin': <class '__main__.VideoPlugin'>}
{'MyPlugin': <class '__main__.MyPlugin'>}

Key Points

•type is the default metaclass; classes are instances of type
•type(name, bases, dict) creates classes dynamically
•Custom metaclass __new__ is called when a class is created (not instantiated)
•__init_subclass__ (3.6+) is a simpler alternative for most metaclass use cases
•Metaclasses can validate, register, or modify classes at creation time

Gotchas

•Metaclasses are powerful but rarely needed; __init_subclass__ handles most cases
•Metaclass conflicts: a class cannot have two unrelated metaclasses
•type.__new__ creates the class; type.__init__ initializes it; type.__call__ creates instances

Metaclasses are 'classes of classes.' type is the default metaclass. Custom metaclasses override __new__ to control class creation. __init_subclass__ (3.6+) is a simpler alternative for most cases.

•type is the metaclass of all classes; custom metaclasses control class creation
•__init_subclass__ is simpler and preferred for most cases

# type creates classes
MyClass = type('MyClass', (), {'x': 42})

# Custom metaclass
class RegistryMeta(type):
    _registry = {}
    def __new__(mcs, name, bases, ns):
        cls = super().__new__(mcs, name, bases, ns)
        if bases:
            mcs._registry[name] = cls
        return cls

# Simpler: __init_subclass__
class Base:
    def __init_subclass__(cls): print(f'Registered: {cls.__name__}')

Metaclasses are rarely needed; __init_subclass__ handles most use cases

Question 6 / 15

How does Python's attribute lookup chain work (MRO and descriptor protocol)?

When you access obj.attr, Python follows a specific lookup order:

Data descriptors from the class (and its MRO): If type(obj).__mro__ has a data descriptor for attr, it wins.
Instance __dict__: obj.__dict__['attr'].
Non-data descriptors and class attributes: From the class and its MRO.
__getattr__: Called as a fallback if all above fail.

The MRO (Method Resolution Order) determines the order in which base classes are searched. Python uses the C3 linearization algorithm to compute it.

__getattribute__ is called for EVERY attribute access; __getattr__ is called only when normal lookup fails.

Pseudo Code

obj.attr lookup:
1. type(obj).__mro__ -> find data descriptor for attr -> call __get__
2. obj.__dict__['attr'] -> return value
3. type(obj).__mro__ -> find non-data descriptor or class attr
4. __getattr__(self, 'attr') -> fallback
5. Raise AttributeError

Code

# MRO: C3 linearization
class A:
    x = 'A'

class B(A):
    pass

class C(A):
    x = 'C'

class D(B, C):
    pass

print(D.__mro__)
# (<class 'D'>, <class 'B'>, <class 'C'>, <class 'A'>, <class 'object'>)
print(D.x)  # 'C' (B has no x, so C is next in MRO)

# Attribute lookup demonstration
class DataDesc:
    """Data descriptor (has __set__)."""
    def __get__(self, obj, cls):
        print("Data descriptor __get__")
        return 'data_desc_value'
    def __set__(self, obj, value):
        print(f"Data descriptor __set__({value})")

class NonDataDesc:
    """Non-data descriptor (only __get__)."""
    def __get__(self, obj, cls):
        print("Non-data descriptor __get__")
        return 'non_data_desc_value'

class MyClass:
    data_attr = DataDesc()
    non_data_attr = NonDataDesc()

obj = MyClass()

# Data descriptor wins over instance __dict__
obj.__dict__['data_attr'] = 'instance_value'
print(obj.data_attr)  # "Data descriptor __get__" -> 'data_desc_value'
# Data descriptor takes priority!

# Instance __dict__ wins over non-data descriptor
obj.__dict__['non_data_attr'] = 'instance_value'
print(obj.non_data_attr)  # 'instance_value' (NOT the descriptor)

# __getattribute__ vs __getattr__
class Logged:
    def __init__(self):
        self.x = 10

    def __getattribute__(self, name):
        """Called for EVERY attribute access."""
        print(f"__getattribute__({name})")
        return super().__getattribute__(name)

    def __getattr__(self, name):
        """Called only when normal lookup FAILS."""
        print(f"__getattr__({name})")
        return f"default_{name}"

obj = Logged()
print(obj.x)        # __getattribute__(x) -> 10
print(obj.missing)  # __getattribute__(missing) -> __getattr__(missing) -> 'default_missing'

(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
C
Data descriptor __get__
data_desc_value
instance_value
__getattribute__(x)
10
__getattribute__(missing)
__getattr__(missing)
default_missing

Key Points

•Lookup order: data descriptor > instance __dict__ > non-data descriptor > class attrs
•MRO uses C3 linearization; view with Class.__mro__
•__getattribute__ is called for every access; __getattr__ only on failure
•Data descriptors (with __set__) always override instance __dict__
•property, classmethod, staticmethod are data descriptors

Gotchas

•Overriding __getattribute__ without calling super() breaks all attribute access
•Data vs non-data descriptor priority difference catches many people off guard
•Calling self.x inside __getattribute__ causes infinite recursion; use super().__getattribute__

Attribute lookup: data descriptor > instance __dict__ > non-data descriptor > class attrs > __getattr__. MRO (C3 linearization) determines base class search order. __getattribute__ is called for every access; __getattr__ only on failure.

•Data descriptor > instance __dict__ > non-data descriptor
•MRO via C3 linearization; __getattr__ is the fallback

class D(B, C):
    pass

print(D.__mro__)  # D -> B -> C -> A -> object (C3 linearization)

# Data descriptor beats instance __dict__
class Desc:
    def __get__(self, obj, cls): return 'desc'
    def __set__(self, obj, val): pass  # Makes it a data descriptor

class MyClass:
    attr = Desc()

obj = MyClass()
obj.__dict__['attr'] = 'instance'
print(obj.attr)  # 'desc' (data descriptor wins!)

Infinite recursion if __getattribute__ calls self.x without super()

Question 7 / 15

What is string interning and small integer caching in CPython?

CPython pre-allocates and caches certain commonly used objects for performance:

Small integer caching: Integers from -5 to 256 are pre-allocated at interpreter startup. Any reference to these values returns the same object (identity-equal with is).

String interning: Short strings that look like identifiers (alphanumeric + underscore) are automatically interned (stored once and reused). You can also manually intern strings with sys.intern().

Other cached objects: None, True, False, empty tuple (), empty string '', empty bytes b'' are all singletons.

These are implementation details of CPython, not guaranteed by the Python language specification.

Pseudo Code

Integers -5 to 256 are pre-cached (same object in memory)
Identifier-like strings are interned (single copy shared)
None, True, False are singletons
These are CPython implementation details, not spec guarantees

Code

import sys

# Small integer caching: -5 to 256
a = 256
b = 256
print(a is b)  # True (same object, pre-cached)

c = 257
d = 257
print(c is d)  # False (different objects, outside cache)
# Note: in the REPL or same expression, CPython may optimize this differently

# Verify with id()
print(id(256), id(256))  # Same id
print(id(257), id(257))  # May differ

# String interning
s1 = 'hello'
s2 = 'hello'
print(s1 is s2)  # True (auto-interned: looks like identifier)

s3 = 'hello world'
s4 = 'hello world'
print(s3 is s4)  # May be False (space makes it not identifier-like)
# Note: CPython may intern compile-time constants in the same module

# Manual interning
s5 = sys.intern('hello world')
s6 = sys.intern('hello world')
print(s5 is s6)  # True (manually interned)

# Singleton objects
print(True is True)    # Always True
print(None is None)    # Always True
print(() is ())        # True (empty tuple is cached)
print('' is '')        # True (empty string is cached)

# Why this matters: always use == for value comparison
x = 300
y = 300
print(x == y)   # True (values are equal)
print(x is y)   # Unpredictable! (depends on CPython optimization)

# is should ONLY be used for None, True, False checks
result = None
if result is None:    # Correct: use 'is' for None
    print("No result")
if result == None:    # Works but less Pythonic
    print("No result")

# Peephole optimization: constant folding
a = 'hello' + ' ' + 'world'  # Folded at compile time to 'hello world'
b = 'hello world'
print(a is b)  # True (same constant)

True
True
4388716192 4388716192
4461410128 4461410128
True
True
True
True
True
True
True
True
True
No result
No result
True

Key Points

•Integers -5 to 256 are pre-cached singletons in CPython
•Identifier-like strings are auto-interned; sys.intern() for manual interning
•None, True, False, empty tuple, empty string are singletons
•Always use == for value comparison; is only for None/True/False
•These are CPython implementation details, not Python language guarantees

Gotchas

•Never use 'is' for integer or string comparison; use '==' for values
•Interning behavior varies across Python versions and implementations
•REPL and .py files may behave differently due to compile-time optimizations

CPython caches integers -5 to 256 and interns short identifier-like strings. None, True, False are singletons. These are CPython implementation details. Always use == for comparison, is only for None.

•Integers -5 to 256 are cached; identifier strings are interned
•Use == for values; is only for None/True/False

a = 256; b = 256
print(a is b)   # True (cached)

c = 257; d = 257
print(c is d)   # Unpredictable!

# Always use == for values
print(c == d)   # True (reliable)

# 'is' only for None
if result is None: ...

Never rely on 'is' for integer or string comparison

Question 8 / 15

How do you profile Python code for performance optimization?

Python provides several profiling tools to identify performance bottlenecks:

timeit: Micro-benchmarking individual expressions or functions.
cProfile: Deterministic profiling of function call counts and times.
line_profiler: Line-by-line profiling (external package).
memory_profiler: Line-by-line memory usage (external package).
py-spy: Sampling profiler that works on running processes without modification.

Rule of thumb: Profile first, optimize later. Never guess where the bottleneck is.

Pseudo Code

1. timeit for micro-benchmarks (which approach is faster?)
2. cProfile for overall function-level profiling
3. line_profiler for line-by-line hotspot analysis
4. memory_profiler for memory leak detection
5. Always profile before optimizing

Code

import timeit
import cProfile
import pstats

# 1. timeit: micro-benchmarking
# Compare list comprehension vs loop
result = timeit.timeit('[i**2 for i in range(100)]', number=100_000)
print(f"List comp: {result:.3f}s")

result = timeit.timeit(
    """
result = []
for i in range(100):
    result.append(i**2)
""", number=100_000)
print(f"Loop: {result:.3f}s")

# timeit with setup
result = timeit.timeit(
    'sorted(data)',
    setup='import random; data = [random.randint(0, 1000) for _ in range(100)]',
    number=10_000
)
print(f"Sort: {result:.3f}s")

# 2. cProfile: function-level profiling
def slow_function():
    total = 0
    for i in range(1_000_000):
        total += i ** 2
    return total

def main():
    slow_function()
    sorted(range(100_000, 0, -1))

# Profile with cProfile
cProfile.run('main()')
#          3 function calls in 0.123 seconds
#    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
#         1    0.100    0.100    0.100    0.100 <string>:1(slow_function)
#         1    0.023    0.023    0.023    0.023 {built-in method builtins.sorted}

# Save profile to file and analyze
with cProfile.Profile() as profiler:
    main()

stats = pstats.Stats(profiler)
stats.sort_stats('cumulative')
stats.print_stats(10)  # Top 10 functions by cumulative time

# 3. line_profiler (pip install line_profiler)
# @profile  # Decorator added by kernprof
# def slow_function():
#     total = 0                    # 0.001s
#     for i in range(1_000_000):   # 0.050s
#         total += i ** 2          # 0.049s
#     return total
# Run: kernprof -l -v script.py

# 4. Quick timing with context manager
import time
from contextlib import contextmanager

@contextmanager
def timer(name=""):
    start = time.perf_counter()
    yield
    elapsed = time.perf_counter() - start
    print(f"{name}: {elapsed:.4f}s")

with timer("sort"):
    sorted(range(1_000_000, 0, -1))

with timer("comprehension"):
    [i**2 for i in range(1_000_000)]

# Expected Output: (timing values are approximate)
# List comp: 0.209s
# Loop: 0.225s
# Sort: 0.023s
#          6 function calls in 0.035 seconds
#
#    Ordered by: standard name
#
#    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
#         1    0.000    0.000    0.035    0.035 <string>:1(<module>)
#         1    0.033    0.033    0.033    0.033 <string>:27(slow_function)
#         1    0.000    0.000    0.035    0.035 <string>:33(main)
#         1    0.000    0.000    0.035    0.035 {built-in method builtins.exec}
#         1    0.001    0.001    0.001    0.001 {built-in method builtins.sorted}
#         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
#
#
#          5 function calls in 0.034 seconds
#
#    Ordered by: cumulative time
#
#    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
#         1    0.000    0.000    0.034    0.034 <string>:33(main)
#         1    0.033    0.033    0.033    0.033 <string>:27(slow_function)
#         1    0.001    0.001    0.001    0.001 {built-in method builtins.sorted}
#         1    0.000    0.000    0.000    0.000 .../cProfile.py:119(__exit__)
#         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
#
#
# sort: 0.0132s
# comprehension: 0.0271s

Key Points

•timeit for micro-benchmarks; cProfile for function-level profiling
•line_profiler for line-by-line analysis; memory_profiler for memory
•cProfile.run('func()') prints a profiling report
•pstats.Stats can sort by cumulative time, call count, etc.
•Profile first, optimize later; never guess where the bottleneck is

Gotchas

•timeit disables garbage collection by default; results may not reflect real-world behavior
•cProfile adds overhead; profiled code runs 2-3x slower
•Micro-optimizations rarely matter; focus on algorithmic improvements (O(n) vs O(n^2))

timeit for micro-benchmarks, cProfile for function-level profiling, line_profiler for line-by-line analysis. Profile first, then optimize. Focus on algorithms, not micro-optimizations.

•timeit for benchmarks; cProfile for profiling; profile first, optimize later
•Focus on algorithms (O(n) vs O(n^2)), not micro-optimizations

import timeit
import cProfile

# Micro-benchmark
timeit.timeit('[i**2 for i in range(100)]', number=100_000)

# Function-level profile
cProfile.run('my_function()')

# Quick timer
import time
start = time.perf_counter()
# ... code ...
print(f"{time.perf_counter() - start:.4f}s")

cProfile adds overhead; results are relative, not absolute

Question 9 / 15

What common Python optimization techniques should you know?

Python performance optimization follows the rule: correct first, then profile, then optimize. Common techniques:

Use built-in functions: sum(), min(), max(), sorted() are implemented in C and much faster than Python loops.
List comprehensions over loops: Comprehensions are optimized in CPython.
Local variables: Local lookups are faster than global or attribute lookups.
Avoid repeated attribute lookups: Cache self.method in a local variable in tight loops.
__slots__: Save memory for data-heavy classes.
Generator expressions: Use memory-efficient generators for large datasets.
collections.deque: O(1) appends/pops from both ends (vs O(n) for list.pop(0)).
String joining: ''.join(list) not += in loops.
functools.lru_cache: Cache expensive function calls.

Pseudo Code

1. Profile to find actual bottleneck
2. Use built-ins (C-implemented) over Python loops
3. List comps > for loops > while loops
4. Local vars > global vars > attribute lookups
5. Generators for large datasets (memory)
6. deque for queue operations
7. ''.join() for string building
8. lru_cache for expensive repeated calls

Code

import timeit
from functools import lru_cache
from collections import deque

# 1. Built-ins vs loops
# Slow
def sum_loop(n):
    total = 0
    for i in range(n):
        total += i
    return total

# Fast (C implementation)
def sum_builtin(n):
    return sum(range(n))

# 2. List comprehension vs loop
# Slow
def squares_loop(n):
    result = []
    for i in range(n):
        result.append(i ** 2)
    return result

# Fast
def squares_comp(n):
    return [i ** 2 for i in range(n)]

# 3. Local variable optimization
def slow_method(self, items):
    for item in items:
        self.process(item)  # Attribute lookup each iteration

def fast_method(self, items):
    process = self.process  # Cache attribute lookup
    for item in items:
        process(item)  # Local lookup (faster)

# 4. String joining
# Slow: O(n^2) due to string immutability
def build_string_bad(n):
    result = ''
    for i in range(n):
        result += str(i) + ','  # Creates new string each time
    return result

# Fast: O(n)
def build_string_good(n):
    return ','.join(str(i) for i in range(n))

# 5. deque vs list for queue operations
def queue_list(n):
    q = []
    for i in range(n):
        q.append(i)
    for _ in range(n):
        q.pop(0)  # O(n) per pop!

def queue_deque(n):
    q = deque()
    for i in range(n):
        q.append(i)
    for _ in range(n):
        q.popleft()  # O(1) per pop!

# 6. lru_cache for memoization
@lru_cache(maxsize=128)
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

fibonacci(100)  # Instant with caching; O(2^n) without

# 7. Generator for large data (memory efficient)
def process_large_file(filename):
    # Bad: loads entire file into memory
    # lines = open(filename).readlines()

    # Good: processes line by line
    with open(filename) as f:
        for line in f:  # Generator-based iteration
            yield line.strip()

# 8. Use sets for membership testing
my_list = list(range(10000))
my_set = set(range(10000))

# list: O(n) lookup
# set: O(1) lookup
# timeit: 9999 in my_list -> ~0.1ms
# timeit: 9999 in my_set -> ~0.00001ms

Key Points

•Built-in functions (sum, sorted, min, max) are C-optimized; use them over loops
•List comprehensions are faster than append-in-a-loop
•Cache attribute lookups in local variables for tight loops
•''.join(iterable) is O(n); += in a loop is O(n^2) for strings
•Use sets for membership testing (O(1) vs O(n) for lists)

Gotchas

•Premature optimization is the root of all evil; profile first
•lru_cache only works with hashable arguments; unhashable types cause TypeError
•Generator expressions save memory but can only be iterated once

Use built-ins over loops, comprehensions over append, local vars over globals, ''.join() over +=, sets for membership, deque for queues, lru_cache for memoization. Always profile first.

•Built-ins and comprehensions over manual loops
•Profile first; focus on algorithms, not micro-opts

# Built-in: sum(range(n)) not manual loop
# String: ','.join(items) not += in loop
# Membership: use set not list for 'in' checks
# Queue: deque.popleft() not list.pop(0)
# Memoize: @lru_cache for expensive calls
from functools import lru_cache

@lru_cache(maxsize=128)
def fib(n):
    if n < 2: return n
    return fib(n-1) + fib(n-2)

Premature optimization wastes time; profile to find actual bottlenecks

Question 10 / 15

What is the PyObject structure and how does CPython's object model work?

In CPython, every Python object is a C struct containing at minimum:

ob_refcnt (Py_ssize_t): The reference count for garbage collection.
ob_type (PyTypeObject*): Pointer to the object's type.

This is the PyObject header. Variable-size objects (lists, tuples) additionally have ob_size.

The type object (PyTypeObject) contains:

Method tables (tp_methods)
Slot functions (tp_init, tp_new, tp_repr, tp_hash)
Type name, base classes, dict, etc.

Everything in Python is a PyObject: integers, strings, functions, classes, modules, even type itself.

Pseudo Code

PyObject:
  ob_refcnt: reference count
  ob_type: pointer to type object

PyVarObject (extends PyObject):
  ob_size: number of items (for lists, tuples, etc.)

PyTypeObject:
  tp_name, tp_base, tp_dict, tp_methods, tp_init, tp_new...

Code

import sys
import ctypes

# Every object has a reference count
x = [1, 2, 3]
print(sys.getrefcount(x))  # Usually 2 (x + getrefcount arg)

# Reference count changes
y = x       # refcount += 1
print(sys.getrefcount(x))  # 3
del y       # refcount -= 1
print(sys.getrefcount(x))  # 2

# Object size (including PyObject header)
print(sys.getsizeof(0))        # 28 bytes (int with value 0)
print(sys.getsizeof(1))        # 28 bytes
print(sys.getsizeof(2**30))    # 32 bytes (needs more storage)
print(sys.getsizeof(''))       # 49 bytes (empty string)
print(sys.getsizeof('a'))      # 50 bytes
print(sys.getsizeof([]))       # 56 bytes (empty list)
print(sys.getsizeof(()))       # 40 bytes (empty tuple, smaller than list)

# Object identity (memory address in CPython)
x = 42
print(id(x))           # Memory address of the PyObject
print(hex(id(x)))      # Hex address

# Type object inspection
print(type(42))        # <class 'int'>
print(type(int))       # <class 'type'>
print(type(type))      # <class 'type'> (type is its own type!)

# type has slots for all dunder methods
print(int.__add__)     # Slot wrapper for int.__add__
print(int.__repr__)    # Slot wrapper for int.__repr__

# Verify everything is an object
import types

def func(): pass

print(isinstance(42, object))        # True
print(isinstance('hello', object))   # True
print(isinstance(func, object))      # True
print(isinstance(int, object))       # True
print(isinstance(type, object))      # True
print(isinstance(None, object))      # True

# Object header details
print(sys.getsizeof(object()))  # 16 bytes (smallest possible: refcount + type pointer)

# id() returns the memory address in CPython
a = 'hello'
b = 'hello'
print(id(a) == id(b))  # True (interned)
print(a is b)           # True (same PyObject)

# Expected Output: (object addresses will vary)
# 2
# 3
# 2
# 28
# 28
# 32
# 41
# 42
# 56
# 40
# 4381352928
# 0x...
# <class 'int'>
# <class 'type'>
# <class 'type'>
# <slot wrapper '__add__' of 'int' objects>
# <slot wrapper '__repr__' of 'int' objects>
# True
# True
# True
# True
# True
# True
# 16
# True
# True

Key Points

•Every Python object has ob_refcnt (reference count) and ob_type (type pointer)
•Variable-size objects add ob_size for element count
•id() returns the memory address of the PyObject in CPython
•sys.getsizeof() returns the size in bytes including the PyObject header
•Everything in Python is a PyObject, including type itself

Gotchas

•sys.getsizeof() does not include referenced objects (e.g., list elements)
•id() values should not be used for anything other than identity comparison
•Reference counts include internal CPython references; getrefcount adds 1 for its own argument

Every Python object is a PyObject with a reference count (ob_refcnt) and type pointer (ob_type). Variable-size objects add ob_size. id() returns the memory address. sys.getsizeof() shows object size in bytes.

•PyObject = ob_refcnt + ob_type; everything is a PyObject
•id() = memory address; sys.getsizeof() = object size

import sys

# Every object has refcount and type pointer
x = [1, 2, 3]
print(sys.getrefcount(x))  # Reference count
print(sys.getsizeof(x))    # Size in bytes
print(id(x))               # Memory address
print(type(x))             # Type object

2
88
4375234880
<class 'list'>

sys.getsizeof() excludes referenced objects (shallow size only)

Question 11 / 15

How does the dis module work and what can you learn from Python bytecode?

The dis module disassembles Python bytecode, showing the low-level instructions the Python Virtual Machine executes. This is useful for:

Understanding how Python translates your code
Comparing the efficiency of different approaches
Debugging optimization surprises
Learning CPython internals

Bytecode format:

  line_no    offset  OPCODE          arg  (arg_repr)

Common opcodes:

LOAD_FAST / LOAD_GLOBAL / LOAD_CONST - Load a value onto the stack
STORE_FAST / STORE_GLOBAL - Store a value from the stack
BINARY_ADD / BINARY_MULTIPLY - Arithmetic operations
CALL_FUNCTION - Call a function
RETURN_VALUE - Return from function
JUMP_IF_FALSE_OR_POP - Conditional jump

Pseudo Code

1. dis.dis(func) shows bytecode for a function
2. dis.dis(code_string) shows bytecode for a string
3. Each instruction operates on the evaluation stack
4. LOAD pushes; STORE pops; BINARY_* pops 2 and pushes result
5. Compare approaches by counting and analyzing opcodes

Code

import dis

# Basic function disassembly
def add(a, b):
    return a + b

print("=== add ===")
dis.dis(add)
#   2           0 LOAD_FAST                0 (a)
#               2 LOAD_FAST                1 (b)
#               4 BINARY_ADD
#               6 RETURN_VALUE

# Compare: list comp vs loop
def comp(n):
    return [i ** 2 for i in range(n)]

def loop(n):
    result = []
    for i in range(n):
        result.append(i ** 2)
    return result

print("=== comprehension ===")
dis.dis(comp)

print("=== loop ===")
dis.dis(loop)  # More instructions: LOAD_ATTR for .append, etc.

# Observe constant folding
def constant_folding():
    x = 2 * 3 * 4
    y = 'hello' + ' ' + 'world'
    return x, y

print("=== constant folding ===")
dis.dis(constant_folding)
# LOAD_CONST  1 (24)        -- 2*3*4 computed at compile time
# LOAD_CONST  2 ('hello world')  -- string concatenated at compile time

# Compare local vs global lookup
global_var = 10

def local_lookup():
    x = 10
    return x  # LOAD_FAST (fast, indexed lookup)

def global_lookup():
    return global_var  # LOAD_GLOBAL (slower, dict lookup)

print("=== local ===")
dis.dis(local_lookup)
# LOAD_FAST (index-based, fast)

print("=== global ===")
dis.dis(global_lookup)
# LOAD_GLOBAL (dict-based, slower)

# Get bytecode details programmatically
def example(x):
    if x > 0:
        return x * 2
    return 0

for instr in dis.get_instructions(example):
    print(f"{instr.offset:4d} {instr.opname:20s} {instr.argrepr}")
#    0 LOAD_FAST             x
#    2 LOAD_CONST            0
#    4 COMPARE_OP            >
#    ...etc

# Instruction count comparison
def count_instructions(func):
    return len(list(dis.get_instructions(func)))

print(f"comp instructions: {count_instructions(comp)}")
print(f"loop instructions: {count_instructions(loop)}")

=== add ===
4           0 RESUME                   0

5           2 LOAD_FAST                0 (a)
4 LOAD_FAST                1 (b)
6 BINARY_OP                0 (+)
10 RETURN_VALUE
=== comprehension ===
15           0 RESUME                   0

16           2 LOAD_GLOBAL              1 (NULL + range)
12 LOAD_FAST                0 (n)
14 CALL                     1
22 GET_ITER
24 LOAD_FAST_AND_CLEAR      1 (i)
26 SWAP                     2
28 BUILD_LIST               0
30 SWAP                     2
>>   32 FOR_ITER                 7 (to 50)
36 STORE_FAST               1 (i)
38 LOAD_FAST                1 (i)
40 LOAD_CONST               1 (2)
42 BINARY_OP                8 (**)
46 LIST_APPEND              2
48 JUMP_BACKWARD            9 (to 32)
>>   50 END_FOR
52 SWAP                     2
54 STORE_FAST               1 (i)
56 RETURN_VALUE
>>   58 SWAP                     2
60 POP_TOP
62 SWAP                     2
64 STORE_FAST               1 (i)
66 RERAISE                  0
ExceptionTable:
28 to 50 -> 58 [2]
=== loop ===
18           0 RESUME                   0

19           2 BUILD_LIST               0
4 STORE_FAST               1 (result)

20           6 LOAD_GLOBAL              1 (NULL + range)
16 LOAD_FAST                0 (n)
18 CALL                     1
26 GET_ITER
>>   28 FOR_ITER                22 (to 76)
32 STORE_FAST               2 (i)

21          34 LOAD_FAST                1 (result)
36 LOAD_ATTR                3 (NULL|self + append)
56 LOAD_FAST                2 (i)
58 LOAD_CONST               1 (2)
60 BINARY_OP                8 (**)
64 CALL                     1
72 POP_TOP
74 JUMP_BACKWARD           24 (to 28)

20     >>   76 END_FOR

22          78 LOAD_FAST                1 (result)
80 RETURN_VALUE
=== constant folding ===
31           0 RESUME                   0

32           2 LOAD_CONST               1 (24)
4 STORE_FAST               0 (x)

33           6 LOAD_CONST               2 ('hello world')
8 STORE_FAST               1 (y)

34          10 LOAD_FAST                0 (x)
12 LOAD_FAST                1 (y)
14 BUILD_TUPLE              2
16 RETURN_VALUE
=== local ===
44           0 RESUME                   0

45           2 LOAD_CONST               1 (10)
4 STORE_FAST               0 (x)

46           6 LOAD_FAST                0 (x)
8 RETURN_VALUE
=== global ===
48           0 RESUME                   0

49           2 LOAD_GLOBAL              0 (global_var)
12 RETURN_VALUE
0 RESUME
2 LOAD_FAST            x
4 LOAD_CONST           0
6 COMPARE_OP           >
10 POP_JUMP_IF_FALSE    to 22
12 LOAD_FAST            x
14 LOAD_CONST           2
16 BINARY_OP            *
20 RETURN_VALUE
22 RETURN_CONST         0
comp instructions: 25
loop instructions: 20

Key Points

•dis.dis(func) shows human-readable bytecode for any function
•LOAD_FAST (local) is faster than LOAD_GLOBAL (global dict lookup)
•Constant folding computes literal expressions at compile time
•Comprehensions generate fewer instructions than equivalent loops
•dis.get_instructions() returns instruction objects for programmatic analysis

Gotchas

•Fewer bytecode instructions does not always mean faster (instruction cost varies)
•Bytecode changes between Python versions; do not write version-dependent code
•dis output is CPython-specific; other implementations have different bytecodes

dis.dis(func) shows Python bytecode instructions. Useful for comparing approaches (LOAD_FAST vs LOAD_GLOBAL), seeing constant folding, and understanding CPython internals. dis.get_instructions() for programmatic access.

•dis.dis() shows bytecode; LOAD_FAST is faster than LOAD_GLOBAL
•Constant folding optimizes literal expressions at compile time

import dis

def add(a, b):
    return a + b

dis.dis(add)
#  LOAD_FAST    0 (a)
#  LOAD_FAST    1 (b)
#  BINARY_ADD
#  RETURN_VALUE

# Programmatic access
for instr in dis.get_instructions(add):
    print(instr.opname, instr.argrepr)

3           0 RESUME                   0

4           2 LOAD_FAST                0 (a)
4 LOAD_FAST                1 (b)
6 BINARY_OP                0 (+)
10 RETURN_VALUE
RESUME
LOAD_FAST a
LOAD_FAST b
BINARY_OP +
RETURN_VALUE

Bytecode changes between Python versions

Question 12 / 15

How does Python's MRO (Method Resolution Order) work internally?

The MRO (Method Resolution Order) determines the order in which base classes are searched when calling a method. Python uses the C3 linearization algorithm (since Python 2.3).

C3 linearization rules:

A class always comes before its parents.
If a class inherits from multiple classes, they maintain their order.
A class appears only once in the MRO.

The algorithm: L[C] = C + merge(L[B1], L[B2], ..., B1, B2, ...) where merge takes the first head that does not appear in the tail of any other list.

super() follows the MRO, not just the parent class, enabling cooperative multiple inheritance.

Pseudo Code

C3 Linearization:
L[C(B1, B2)] = C + merge(L[B1], L[B2], [B1, B2])

merge rule: take the first head that is not in any other list's tail

super() follows MRO, not direct parent
Class.__mro__ shows the computed order

Code

# Simple MRO
class A:
    def method(self):
        return 'A'

class B(A):
    def method(self):
        return 'B'

class C(A):
    def method(self):
        return 'C'

class D(B, C):
    pass

print(D.__mro__)
# (D, B, C, A, object)
# D -> B -> C -> A -> object
print(D().method())  # 'B' (B comes before C in MRO)

# Diamond problem
class Base:
    def method(self):
        print('Base.method')

class Left(Base):
    def method(self):
        print('Left.method')
        super().method()  # Calls Right.method, NOT Base.method!

class Right(Base):
    def method(self):
        print('Right.method')
        super().method()  # Calls Base.method

class Diamond(Left, Right):
    def method(self):
        print('Diamond.method')
        super().method()  # Calls Left.method

print(Diamond.__mro__)
# (Diamond, Left, Right, Base, object)

Diamond().method()
# Diamond.method
# Left.method
# Right.method  <-- super() in Left calls Right, not Base!
# Base.method

# Cooperative multiple inheritance with super()
class Loggable:
    def __init__(self, **kwargs):
        print(f"Loggable.__init__")
        super().__init__(**kwargs)

class Serializable:
    def __init__(self, **kwargs):
        print(f"Serializable.__init__")
        super().__init__(**kwargs)

class User(Loggable, Serializable):
    def __init__(self, name, **kwargs):
        print(f"User.__init__")
        self.name = name
        super().__init__(**kwargs)

print(User.__mro__)
# (User, Loggable, Serializable, object)

u = User(name='Alice')
# User.__init__
# Loggable.__init__
# Serializable.__init__

# Invalid MRO (C3 cannot resolve)
# class X(A, B): pass
# class Y(B, A): pass
# class Z(X, Y): pass  # TypeError: Cannot create consistent MRO

(<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <class 'object'>)
B
(<class '__main__.Diamond'>, <class '__main__.Left'>, <class '__main__.Right'>, <class '__main__.Base'>, <class 'object'>)
Diamond.method
Left.method
Right.method
Base.method
(<class '__main__.User'>, <class '__main__.Loggable'>, <class '__main__.Serializable'>, <class 'object'>)
User.__init__
Loggable.__init__
Serializable.__init__

Key Points

•MRO uses C3 linearization: child before parent, preserve order, no duplicates
•super() follows MRO, not just the direct parent (critical for diamond inheritance)
•Class.__mro__ shows the computed resolution order
•Cooperative multiple inheritance requires all classes to use super().__init__(**kwargs)
•Python raises TypeError if C3 linearization cannot produce a consistent MRO

Gotchas

•super() in Left may call Right.method, not Base.method (follows MRO, not parent)
•All classes in cooperative inheritance must accept and pass **kwargs
•Mixing cooperative (super) and non-cooperative classes causes bugs

MRO uses C3 linearization to order base classes: child before parent, preserve declaration order, no duplicates. super() follows MRO, not just the direct parent. View with Class.__mro__.

•C3 linearization: child first, preserve order, no duplicates
•super() follows MRO, not direct parent

class A: pass
class B(A): pass
class C(A): pass
class D(B, C): pass

print(D.__mro__)  # D -> B -> C -> A -> object

# super() follows MRO
class Left(Base):
    def method(self):
        super().method()  # Calls Right.method (next in MRO), not Base!

super() in one parent may call a sibling, not the grandparent

Question 13 / 15

How do you use __init_subclass and class_getitem__ for class customization?

Python provides hooks for customizing class creation without metaclasses:

__init_subclass__ (PEP 487, Python 3.6+): Called when a class is subclassed. The parent class can validate, register, or modify subclasses.
__class_getitem__ (PEP 560, Python 3.7+): Called when a class is subscripted (e.g., MyClass[int]). Enables generic-like syntax without inheriting from Generic.
__set_name__ (PEP 487, Python 3.6+): Called on descriptors when they are assigned to a class attribute.

These hooks cover most use cases that previously required metaclasses.

Pseudo Code

__init_subclass__: called on the PARENT when a child class is defined
__class_getitem__: called when class is subscripted: MyClass[T]
__set_name__: called on descriptors when assigned to a class attribute

Code

# __init_subclass__: validate and register subclasses
class Plugin:
    _registry = {}

    def __init_subclass__(cls, plugin_name=None, **kwargs):
        super().__init_subclass__(**kwargs)
        name = plugin_name or cls.__name__
        Plugin._registry[name] = cls
        print(f"Registered plugin: {name}")

class AudioPlugin(Plugin, plugin_name='audio'):
    def process(self):
        return "Processing audio"

class VideoPlugin(Plugin):
    def process(self):
        return "Processing video"

print(Plugin._registry)
# {'audio': <class 'AudioPlugin'>, 'VideoPlugin': <class 'VideoPlugin'>}

# __init_subclass__ for validation
class Serializable:
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        if not hasattr(cls, 'serialize'):
            raise TypeError(f"{cls.__name__} must implement serialize()")

class User(Serializable):
    def serialize(self):
        return {'name': self.name}

# class BadUser(Serializable):  # TypeError: must implement serialize()
#     pass

# __class_getitem__: generic-like subscript syntax
class TypedList:
    def __init__(self, item_type):
        self._type = item_type
        self._items = []

    def __class_getitem__(cls, item_type):
        """Called when TypedList[int] is used."""
        def factory():
            return cls(item_type)
        factory.__name__ = f"{cls.__name__}[{item_type.__name__}]"
        return factory

    def append(self, item):
        if not isinstance(item, self._type):
            raise TypeError(f"Expected {self._type.__name__}, got {type(item).__name__}")
        self._items.append(item)

    def __repr__(self):
        return f"TypedList[{self._type.__name__}]({self._items})"

# Usage
int_list = TypedList[int]()
int_list.append(42)
# int_list.append('hello')  # TypeError

# __set_name__: descriptors know their attribute name
class Field:
    def __set_name__(self, owner, name):
        """Called when Field() is assigned to a class attribute."""
        self.name = name
        self.private_name = f'_{name}'
        print(f"Field '{name}' registered on {owner.__name__}")

    def __get__(self, obj, cls):
        if obj is None:
            return self
        return getattr(obj, self.private_name, None)

    def __set__(self, obj, value):
        setattr(obj, self.private_name, value)

class Model:
    name = Field()   # __set_name__ called: "Field 'name' registered on Model"
    email = Field()  # __set_name__ called: "Field 'email' registered on Model"

Registered plugin: audio
Registered plugin: VideoPlugin
{'audio': <class '__main__.AudioPlugin'>, 'VideoPlugin': <class '__main__.VideoPlugin'>}
Field 'name' registered on Model
Field 'email' registered on Model

Key Points

•__init_subclass__ is called on the parent when a child is defined (no metaclass needed)
•Accepts keyword arguments from the class definition: class Child(Parent, key=val)
•__class_getitem__ enables MyClass[T] syntax without Generic inheritance
•__set_name__ tells descriptors what attribute name they are assigned to
•These hooks replace metaclasses for most registration and validation use cases

Gotchas

•__init_subclass__ is NOT called for the class that defines it, only for subclasses
•Always call super().__init_subclass__(**kwargs) to support further subclassing
•__class_getitem__ returns a value (usually a class or parameterized type), not None

__init_subclass__ is called on the parent when subclassed (great for registration/validation). __class_getitem__ enables MyClass[T] syntax. __set_name__ tells descriptors their attribute name. All simpler than metaclasses.

•__init_subclass__ for registration/validation; __class_getitem__ for subscript syntax
•Simpler than metaclasses for most use cases

class Plugin:
    _registry = {}
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        Plugin._registry[cls.__name__] = cls

class AudioPlugin(Plugin): ...  # Auto-registered
print(Plugin._registry)  # {'AudioPlugin': <class>}

{'AudioPlugin': <class '__main__.AudioPlugin'>}

Always call super().__init_subclass__(**kwargs)

Question 14 / 15

What are weak references and when should you use them?

A weak reference is a reference to an object that does not prevent the object from being garbage collected. If the only references to an object are weak references, the garbage collector can reclaim it.

Use cases:

Caches: Cache objects without preventing GC (use WeakValueDictionary).
Observer pattern: Observers do not prevent subjects from being collected.
Circular references: Break reference cycles.
Large object tracking: Monitor objects without keeping them alive.

The weakref module provides ref(), proxy(), WeakValueDictionary, WeakKeyDictionary, and WeakSet.

Pseudo Code

1. weakref.ref(obj) creates a weak reference
2. Call the weak ref to get the object (or None if collected)
3. WeakValueDictionary: values are weak refs (auto-removed on GC)
4. WeakSet: elements are weak refs
5. Callbacks can be registered for when the object is collected

Code

import weakref

# Basic weak reference
class LargeObject:
    def __init__(self, name):
        self.name = name
    def __repr__(self):
        return f"LargeObject({self.name})"
    def __del__(self):
        print(f"GC: {self.name} collected")

obj = LargeObject('data')
weak = weakref.ref(obj)

print(weak())       # LargeObject(data) -- object still alive
print(weak() is obj)  # True -- same object

del obj             # Last strong reference removed
print(weak())       # None -- object was garbage collected

# Weak reference with callback
def on_collected(ref):
    print(f"Object was garbage collected! (ref: {ref})")

obj = LargeObject('important')
weak = weakref.ref(obj, on_collected)
del obj  # Triggers callback: "Object was garbage collected!"

# WeakValueDictionary (cache pattern)
class ExpensiveObject:
    def __init__(self, key):
        self.key = key
        self.data = list(range(10000))  # Large data

cache = weakref.WeakValueDictionary()

def get_or_create(key):
    obj = cache.get(key)
    if obj is None:
        print(f"Creating new object for {key}")
        obj = ExpensiveObject(key)
        cache[key] = obj  # Weak reference stored
    else:
        print(f"Cache hit for {key}")
    return obj

obj1 = get_or_create('item1')  # Creating new
obj2 = get_or_create('item1')  # Cache hit

del obj1, obj2  # No strong references; GC can reclaim
# cache['item1'] is now gone

# WeakSet (observer pattern)
class EventBus:
    def __init__(self):
        self._listeners = weakref.WeakSet()

    def subscribe(self, listener):
        self._listeners.add(listener)

    def emit(self, event):
        for listener in list(self._listeners):  # Copy to avoid mutation during iteration
            listener.on_event(event)

class Listener:
    def __init__(self, name):
        self.name = name
    def on_event(self, event):
        print(f"{self.name} received: {event}")

bus = EventBus()
listener1 = Listener('A')
listener2 = Listener('B')
bus.subscribe(listener1)
bus.subscribe(listener2)

bus.emit('hello')  # A and B receive it

del listener1  # Automatically removed from WeakSet
bus.emit('world')  # Only B receives it

# proxy: transparent weak reference
obj = LargeObject('proxy_target')
proxy = weakref.proxy(obj)
print(proxy.name)  # 'proxy_target' (transparent access)
# del obj
# print(proxy.name)  # ReferenceError: weakly-referenced object no longer exists

# Expected Output: (object addresses will vary)
# LargeObject(data)
# True
# GC: data collected
# None
# GC: important collected
# Object was garbage collected! (ref: <weakref at 0x...; dead>)
# Creating new object for item1
# Cache hit for item1
# B received: hello
# A received: hello
# B received: world
# proxy_target
# GC: proxy_target collected

Key Points

•Weak references do not prevent garbage collection
•weakref.ref() returns a callable; call it to get the object or None
•WeakValueDictionary auto-removes entries when values are GC'd (great for caches)
•WeakSet auto-removes elements when they are GC'd (great for observer pattern)
•weakref.proxy() provides transparent access that raises ReferenceError on dead objects

Gotchas

•Not all objects support weak references; built-in types like int, str, tuple do not
•WeakValueDictionary can change size during iteration; iterate over a copy
•Objects with __slots__ need to include '__weakref__' in slots for weak reference support

Weak references do not prevent garbage collection. Use weakref.ref() for manual weak refs, WeakValueDictionary for caches, WeakSet for observer patterns. Objects are auto-removed when garbage collected.

•Weak refs don't prevent GC; WeakValueDictionary for auto-cleaning caches
•WeakSet for observer patterns (auto-unsubscribe on GC)

import weakref

obj = MyClass()
weak = weakref.ref(obj)
print(weak())  # MyClass instance
del obj
print(weak())  # None (collected)

# Cache that auto-cleans
cache = weakref.WeakValueDictionary()
cache['key'] = some_object  # Removed when some_object is GC'd

int, str, tuple do not support weak references

Question 15 / 15

How does CPython's garbage collector handle reference cycles?

CPython uses two garbage collection mechanisms:

Reference counting (primary): Each object tracks how many references point to it. When the count reaches zero, the object is immediately deallocated. This handles most cases.
Cyclic garbage collector (supplementary): Detects and collects reference cycles (A -> B -> A) that reference counting cannot handle. It runs periodically.

The cyclic GC:

Tracks objects that can contain references to other objects (lists, dicts, instances, etc.).
Organizes objects into 3 generations (0, 1, 2) based on how long they have survived.
Generation 0 is collected most frequently (young objects die fast).
Uses a mark-and-sweep-like algorithm to find unreachable cycles.

Pseudo Code

Reference counting: ob_refcnt tracks references; deallocates at 0
Cyclic GC: detects A->B->A cycles that refcount misses
Generational: gen 0 (frequent), gen 1 (less), gen 2 (rare)
Thresholds: (700, 10, 10) by default
gc.collect() forces a full collection

Code

import gc
import sys

# Reference counting
a = [1, 2, 3]
print(sys.getrefcount(a))  # 2 (a + getrefcount arg)
b = a
print(sys.getrefcount(a))  # 3 (a + b + getrefcount arg)
del b
print(sys.getrefcount(a))  # 2 (back to a + getrefcount arg)

# Reference cycle that defeats reference counting
class Node:
    def __init__(self, name):
        self.name = name
        self.ref = None
    def __del__(self):
        print(f"GC: {self.name} collected")

# Create a cycle
a = Node('A')
b = Node('B')
a.ref = b  # A -> B
b.ref = a  # B -> A (cycle!)

del a, b  # refcount does NOT reach 0 due to cycle
# The cyclic GC will eventually collect these

# Force collection
collected = gc.collect()
print(f"Collected {collected} objects")

# GC configuration
print(gc.get_threshold())  # (700, 10, 10)
# Gen 0 collected every 700 allocations - 700 deallocations
# Gen 1 collected every 10 gen-0 collections
# Gen 2 collected every 10 gen-1 collections

# Tune thresholds
gc.set_threshold(1000, 15, 15)  # Less frequent collection

# GC statistics
print(gc.get_stats())
# [{'collections': N, 'collected': N, 'uncollectable': N}, ...]

# Disable/enable GC
gc.disable()  # Disable cyclic GC (reference counting still works)
# ... performance-critical section ...
gc.enable()   # Re-enable

# Find objects in a generation
print(gc.get_count())  # (gen0_count, gen1_count, gen2_count)

# Detect uncollectable objects (with __del__ in cycles - pre 3.4)
# Python 3.4+ can collect cycles even with __del__ (PEP 442)

# Weak references to break cycles
import weakref

class Parent:
    def __init__(self):
        self.children = []

class Child:
    def __init__(self, parent):
        self.parent = weakref.ref(parent)  # Weak ref breaks the cycle

parent = Parent()
child = Child(parent)
parent.children.append(child)

del parent  # parent can be collected; child.parent() returns None

# gc.callbacks for monitoring
def gc_callback(phase, info):
    if phase == 'start':
        print(f"GC starting: gen {info['generation']}")
    elif phase == 'stop':
        print(f"GC done: collected {info['collected']}")

gc.callbacks.append(gc_callback)

2
3
2
GC: A collected
GC: B collected
Collected 2 objects
(700, 10, 10)
[{'collections': 9, 'collected': 141, 'uncollectable': 0}, {'collections': 0, 'collected': 0, 'uncollectable': 0}, {'collections': 1, 'collected': 2, 'uncollectable': 0}]
(8, 0, 0)
GC starting: gen 2
GC done: collected 0

Key Points

•Reference counting is primary (immediate deallocation at refcount 0)
•Cyclic GC handles cycles that refcounting cannot (A->B->A)
•Three generations: gen 0 (frequent), gen 1 (less), gen 2 (rare)
•gc.collect() forces collection; gc.get_threshold() shows collection frequency
•Use weakref to break cycles intentionally (parent-child patterns)

Gotchas

•Disabling GC (gc.disable) only disables the cyclic collector; refcounting still works
•Objects with __del__ in cycles were uncollectable before Python 3.4 (PEP 442 fixed this)
•gc.collect() can cause latency spikes; consider tuning thresholds for real-time applications

CPython uses reference counting (immediate dealloc at 0) plus a cyclic GC (detects A->B->A cycles). Three generations: young objects collected frequently, old objects rarely. Use weakref to break cycles intentionally.

•Refcount for most objects; cyclic GC for reference cycles
•3 generations; use weakref to break intentional cycles

import gc

# Force collection
gc.collect()

# Check thresholds
print(gc.get_threshold())  # (700, 10, 10)

# Break cycles with weakref
import weakref
class Child:
    def __init__(self, parent):
        self.parent = weakref.ref(parent)  # Weak ref, no cycle

(700, 10, 10)

gc.disable() only disables the cyclic collector, not reference counting

Built by onllm.dev

Home GitHub Support awesomePrep

What is CPython and how does the Python compilation process work?

What is the GIL (Global Interpreter Lock) and how does it affect Python performance?

What are __slots__ and how do they optimize memory usage?

What are descriptors in Python and how do __get__, __set__, and __delete__ work?

What are metaclasses in Python and how does type() work?

How does Python's attribute lookup chain work (MRO and descriptor protocol)?

What is string interning and small integer caching in CPython?

How do you profile Python code for performance optimization?

What common Python optimization techniques should you know?

What is the PyObject structure and how does CPython's object model work?

How does the dis module work and what can you learn from Python bytecode?

How does Python's MRO (Method Resolution Order) work internally?

How do you use __init_subclass__ and __class_getitem__ for class customization?

What are weak references and when should you use them?

How does CPython's garbage collector handle reference cycles?

Python Internals & Performance

What are slots and how do they optimize memory usage?

What are descriptors in Python and how do get, set, and delete work?

How do you use __init_subclass and class_getitem__ for class customization?