Context managers and the with statement in Python Backend

We've all shipped code that leaves database connections hanging or file handles unclosed when an exception occurs. Context managers and the with statement in Python backend development are your safety net—they guarantee cleanup of files, locks and connections, no matter how your code exits. Understanding how to use and write context managers separates code that works from code that survives production.

How with calls __enter__ and __exit__

The with statement is syntactic sugar that calls two special methods: __enter__ when execution enters the block, and __exit__ when it leaves. Any object implementing these methods is a context manager. When you write with resource as var:, Python calls resource.__enter__(), binds its return value to var, then guarantees that resource.__exit__() runs before leaving the block—whether by normal completion, exception, or early return.

Consider a database connection pool in a typical e-commerce service. Without context managers, acquiring and releasing connections becomes error-prone:

# Without context manager - easy to leak
conn = pool.acquire()
cursor = conn.cursor()
cursor.execute("SELECT * FROM orders WHERE user_id = %s", (user_id,))
result = cursor.fetchall()
conn.release()  # What if we returned early? Forgot this line?

With a context manager, release happens automatically:

class PooledConnection:
    def __init__(self, pool):
        self.pool = pool
        self.conn = None
    
    def __enter__(self):
        self.conn = self.pool.acquire()
        return self.conn
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        self.conn.release()
        return False  # Don't suppress exceptions

# Usage
with PooledConnection(pool) as conn:
    cursor = conn.cursor()
    cursor.execute("SELECT * FROM orders WHERE user_id = %s", (user_id,))
    return cursor.fetchall()
    # Connection released automatically

The pattern is simple: __enter__ acquires the resource and returns it (often self), __exit__ releases it. The three parameters to __exit__—exception type, value, and traceback—let you inspect what went wrong. Return True to suppress the exception (rarely needed), or False to let it propagate.

Cleanup runs even when exceptions happen

The real power of context managers emerges when exceptions occur. In production, your code will fail in ways you didn't anticipate. File systems fill up, databases become unavailable, network timeouts happen. Context managers ensure cleanup code runs regardless.

Let's look at a real scenario from a trading system where we need to acquire a distributed lock before updating order state:

import redis
import time

class RedisLock:
    def __init__(self, redis_client, key, timeout=10):
        self.redis = redis_client
        self.key = key
        self.timeout = timeout
        self.token = None
    
    def __enter__(self):
        self.token = str(time.time())
        # Spin until we acquire the lock
        while not self.redis.set(self.key, self.token, nx=True, ex=self.timeout):
            time.sleep(0.01)
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        # Always release, even if an exception occurred
        if self.redis.get(self.key) == self.token:
            self.redis.delete(self.key)
        return False

# Usage in order processing
def process_order(order_id):
    with RedisLock(redis_client, f"order:{order_id}"):
        order = get_order(order_id)
        if order.payment_status == "pending":
            # What if this raises an exception?
            process_payment(order)
            order.status = "processing"
            order.save()
        # Lock released here, even if process_payment() threw

Without this guarantee, a crashed payment processor would deadlock subsequent requests. With context managers, the lock releases whether processing succeeds or fails. This is especially critical in financial systems, ride-hailing platforms, or anywhere concurrent resource access matters.

File handling is another classic case. Opening files without context managers is dangerous:

# Risk: file not closed if exception occurs
f = open('export.csv', 'w')
writer = csv.writer(f)
writer.writerow(['user_id', 'email', 'created_at'])
for user in get_all_users():  # What if this fails?
    writer.writerow([user.id, user.email, user.created_at])
f.close()  # Never reached if get_all_users() throws

# Safe: file closed in all cases
with open('export.csv', 'w') as f:
    writer = csv.writer(f)
    writer.writerow(['user_id', 'email', 'created_at'])
    for user in get_all_users():
        writer.writerow([user.id, user.email, user.created_at])
    # File closed here automatically

Write your own with contextlib.contextmanager

Implementing __enter__ and __exit__ works, but for simple cases, the contextlib.contextmanager decorator provides a cleaner syntax. You write a generator function: code before the yield runs on entry, code after runs on exit.

Here's a practical example for timing operations in a SaaS backend:

from contextlib import contextmanager
import time
import logging

@contextmanager
def time_operation(operation_name):
    start = time.perf_counter()
    try:
        yield
    finally:
        elapsed = time.perf_counter() - start
        logging.info(f"{operation_name} took {elapsed:.3f}s")

# Usage
with time_operation("user_signup"):
    user = create_user(email, password)
    send_verification_email(user)
    log_analytics(user_id=user.id, event="signup")
    # Logs "user_signup took 0.456s"

The finally block ensures the timer always logs, whether the block succeeds or an exception occurs. Compare this to a class-based context manager—for simple wrapping logic, the decorator is more readable.

Here's a more complex example that yields a value, useful in a travel booking system:

@contextmanager
def database_transaction(db_connection):
    cursor = db_connection.cursor()
    try:
        yield cursor
        db_connection.commit()
    except Exception:
        db_connection.rollback()
        raise
    finally:
        cursor.close()

# Usage
with database_transaction(db) as cursor:
    cursor.execute("UPDATE bookings SET status = %s WHERE id = %s", ("confirmed", booking_id))
    # Automatic commit on success, rollback on exception

The decorator handles exception propagation automatically. If an exception occurs in the with block, it's raised after the generator resumes from the yield. Your cleanup code can catch it or let it propagate.

Async variants with async with

Modern Python backends increasingly use async/await. Context managers have async equivalents: async with works with objects implementing __aenter__ and __aexit__, and contextlib.asynccontextmanager decorates async generators.

In a ride-hailing service handling concurrent requests, managing database pools asynchronously is essential:

import asyncpg

class AsyncPoolConnection:
    def __init__(self, pool):
        self.pool = pool
        self.conn = None
    
    async def __aenter__(self):
        self.conn = await self.pool.acquire()
        return self.conn
    
    async def __aexit__(self, exc_type, exc_val, exc_tb):
        await self.pool.release(self.conn)
        return False

# Usage
async def get_driver_location(driver_id):
    async with AsyncPoolConnection(db_pool) as conn:
        row = await conn.fetchrow(
            "SELECT lat, lng FROM drivers WHERE id = $1",
            driver_id
        )
        return row

The async variants follow the same pattern: __aenter__ acquires and returns the resource (using await), __aexit__ releases it. This scales well in high-concurrency scenarios where dozens of async tasks might be acquiring and releasing resources simultaneously.

Using the decorator approach for async:

from contextlib import asynccontextmanager

@asynccontextmanager
async def websocket_session(client_id):
    ws = await create_websocket(client_id)
    try:
        yield ws
    finally:
        await ws.close()
        logging.info(f"Closed WebSocket for client {client_id}")

# Usage in a real-time notification handler
async def handle_order_update(order_id, client_id):
    async with websocket_session(client_id) as ws:
        order = await fetch_order(order_id)
        await ws.send_json({"order": order.to_dict()})
        # WebSocket closed automatically

Common mistakes

• Returning True from __exit__ unintentionally: Returning True suppresses exceptions. Unless you're deliberately handling specific exceptions, always return False or omit the return statement.
• Nested context managers without care: When nesting multiple with statements, exceptions in inner blocks still propagate through outer __exit__ methods. Test exception paths carefully.
• Forgetting async/await: Using a regular context manager (class with __enter__/__exit__) inside an async with won't work. You need __aenter__/__aexit__ or an @asynccontextmanager.
• Resource acquisition in __exit__: __exit__ should release resources, not acquire them. Acquiring in __exit__ defeats the purpose and complicates exception handling.
• Ignoring exceptions in cleanup: If your __exit__ or cleanup code raises an exception while handling another exception, the original exception is lost. Log cleanup errors and avoid raising new exceptions in __exit__.

Key takeaways

• Context managers guarantee cleanup code runs via __enter__ and __exit__, protecting against resource leaks even when exceptions occur.
• For simple wrapping logic, use @contextlib.contextmanager decorator over defining a full class—it's cleaner and less error-prone.
• Use async variants (async with, __aenter__/__aexit__, @asynccontextmanager) in concurrent backends to safely manage pools, connections, and WebSockets.
• Always return False from __exit__ unless you're deliberately suppressing specific exceptions; usually you want exceptions to propagate.
• Test exception paths: verify cleanup runs when your with block raises, returns early, or breaks—don't assume happy paths are the only ones that matter in production.