Upgrading Python versions safely: lessons from production

Upgrading Python versions safely: lessons from production

Python releases happen on a predictable schedule, and each one promises performance gains, security patches, and new language features. Yet many teams delay upgrades until forced by deprecated library support or end-of-life deadlines. Upgrading Python versions safely: lessons from production is about building a deliberate process that keeps your stack current without the drama of emergency migrations or weekend outages.

We've managed upgrades across trading platforms, e-commerce backends, and SaaS products handling millions of requests. What separates smooth transitions from production incidents is preparation, testing discipline, and respect for the interconnected nature of Python ecosystems. This article walks through the strategies that work.

Test against new versions in CI early

The earliest signal that an upgrade will cause problems is in your continuous integration pipeline, not in production. By running tests against upcoming Python versions months before you plan to deploy, you catch incompatibilities when they're cheapest to fix.

Set up a matrix in your CI configuration that tests against your current version and the next two minor versions. For example, if you're on Python 3.11, test against 3.11, 3.12, and the development version of 3.13. This isn't about passing every test—it's about flagging deprecation warnings and real failures early.

Here's a GitHub Actions example that tests multiple Python versions:

name: Test Matrix
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.11', '3.12', '3.13-dev']
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -e ".[dev]"
      - run: pytest --tb=short
      - run: python -W error::DeprecationWarning -m pytest

The last step—running tests with deprecation warnings as errors—is crucial. It forces you to address warnings now rather than discover them as failures in a future release. Many teams skip this and regret it later.

In one SaaS product we worked on, treating deprecation warnings as errors caught that a logging middleware was using a function removed in Python 3.12. The warning appeared for months in CI output but was ignored until the actual upgrade attempt. Had we made warnings fail the build, we'd have fixed it incrementally.

Check dependency compatibility

Your code might be perfectly compatible with a new Python version, but your dependencies might not be. Pinned versions lock you to old code that doesn't support newer releases. Checking compatibility before committing to an upgrade saves weeks of debugging.

Before upgrading, audit your dependencies against the target Python version. Use the python-requires metadata in PyPI to quickly see what versions each package supports. Tools like pip-audit check security, but you need to manually scan compatibility for new version support.

For a typical project, create a requirements check script:

#!/usr/bin/env python3
import subprocess
import json
import sys

target_version = sys.argv[1] if len(sys.argv) > 1 else "3.12"

with open("requirements.txt") as f:
    packages = [line.strip() for line in f if line.strip() and not line.startswith("#")]

incompatible = []
for package in packages:
    # Parse package name and version if pinned
    pkg_name = package.split("==")[0].split(">=")[0].split("<")[0].strip()
    
    try:
        result = subprocess.run(
            ["pip", "index", "versions", pkg_name],
            capture_output=True,
            text=True,
            timeout=5
        )
        # Check against PyPI metadata for python_requires
        print(f"Checking {pkg_name}...")
    except Exception as e:
        print(f"Error checking {pkg_name}: {e}")

print("\nManually verify python_requires on PyPI for compatibility.")

More practically, upgrade your dependencies first in a non-production environment. Run your full test suite. If tests pass with newer dependency versions on your current Python release, you've validated that the ecosystem supports them.

In one e-commerce platform, a pinned version of SQLAlchemy didn't support Python 3.12. Rather than wait for a manual upgrade decision, the team created a separate branch, unpinned the version floor, ran tests, and confirmed that the latest version worked. This identified the blocker weeks before the production upgrade window.

Upgrade one minor version at a time

Jumping from Python 3.10 to 3.12 in a single step is risky. Language behavior changes, standard library modules shift, and deprecations accumulate. Upgrading one minor version at a time—3.10 to 3.11, then 3.11 to 3.12—lets you isolate which version introduced a problem.

This is about production risk reduction. If you upgrade two versions and something breaks, you now have twice as many release notes to read and twice as many breaking changes to hunt through.

Plan your upgrade schedule with realistic timelines:

1. Month 1: Test on 3.11, fix deprecation warnings, audit dependencies.
2. Month 2: Deploy 3.11 to staging, run production-like workloads for two weeks.
3. Month 3: Canary deploy 3.11 to 10% of production, monitor metrics for three days.
4. Month 4: Full rollout of 3.11.
5. Repeat the cycle for 3.12.

This staged approach means you're never making two large bets at once. If 3.11 causes issues, you roll back to 3.10. Once 3.11 is stable across the fleet, you repeat the process for 3.12.

Automation is essential here. Use configuration management to track which Python version is deployed where. Tools like Ansible, Terraform, or Docker tags make it trivial to pin and adjust versions across environments without manual SSH sessions.

Each release brings speed improvements

Beyond compatibility and stability, each Python version release includes performance work that often translates directly to reduced CPU usage and latency. These aren't marginal gains—they compound.

Python 3.10 introduced pattern matching and structural improvements. Python 3.11 brought a 10–60% speedup on many workloads through inlined function calls and optimized bytecode. Python 3.12 added the JIT compiler framework (in preview) and further inlining. Python 3.13 continues this trend with object header compression and faster method caching.

To measure these gains in your own code, benchmark key paths before and after the upgrade. A simple approach:

import timeit
import sys

setup = """
def compute(n):
    return sum(i ** 2 for i in range(n))
"""

result = timeit.timeit("compute(10000)", setup=setup, number=100000)
print(f"Python {sys.version_info.major}.{sys.version_info.minor}: {result:.4f}s")

Run this script on both Python versions to see real gains. In one ride-hailing backend we optimized, upgrading from 3.11 to 3.12 reduced p99 latency by 8% on the request routing service without any code changes. That translated to fewer timeouts and a better user experience.

However, don't let performance expectations drive premature upgrades. Stability and correctness come first. Performance is the bonus you earn after a successful, well-tested migration.

Common mistakes

Skipping deprecation warnings in CI: Warnings that you ignore today become errors tomorrow. Make warnings fail the build on future Python versions, or at minimum, log them as explicit tech debt items that block upgrades.

Upgrading all dependencies at once: Combining a Python upgrade with major version bumps of core dependencies (Django, FastAPI, SQLAlchemy) makes debugging impossible. Upgrade Python, then upgrade dependencies one at a time once Python stability is proven.

Testing only on Linux in CI, deploying to mixed environments: If you run Alpine Linux in CI but deploy to Ubuntu, or test on x86 but deploy to ARM, you'll discover environment-specific failures in production. Test the exact OS and architecture you deploy to.

Ignoring typing and linting during upgrades: New Python versions often improve type checking, and stricter linters catch new patterns. Upgrading without re-running mypy and pylint means you miss opportunities to improve code quality alongside the version bump.

Canary deploys that are too small or too short: A 1% canary for one hour won't catch issues that only appear under specific load patterns or with certain data distributions. Aim for at least 5–10% of traffic for at least 24 hours before full rollout.

Key takeaways

• Test against new Python versions in CI months before upgrading production; treat deprecation warnings as build failures.
• Check dependency compatibility against your target Python version before committing to an upgrade plan.
• Upgrade one minor version at a time, with staged deployments: test → staging → canary → full rollout.
• Measure performance improvements after upgrades to justify the effort and catch regressions early.
• Combine Python upgrades with security patching but separate them from major dependency version bumps to isolate failures.
• Automate version management through infrastructure code; avoid manual version changes on individual servers.
• Plan upgrades around your release calendar, not emergency support demands; upgrading under pressure leads to mistakes.