PyO3: Rust in Python — a practical guide for developers

The performance gap that won't go away

Python is fast to write but slow to run. When your application hits a computational bottleneck—processing millions of data points, running ML inference, or crunching financial calculations—a single Python loop can become your bottleneck. This is where PyO3: Rust in Python — a practical guide for developers becomes invaluable. Instead of rewriting your entire system in a compiled language, you can write just the performance-critical portions in Rust and call them seamlessly from Python, keeping your development velocity while reclaiming the speed you need.

Write Python extensions in Rust

PyO3 is a Rust crate that lets you write Python modules and classes entirely in Rust. It handles the complexity of the Python C API so you don't have to. The bridge between Rust and Python is remarkably smooth: you write idiomatic Rust code, annotate it with PyO3 macros, and expose it as a normal Python module that your existing code can import and use.

Let's walk through a practical example. Suppose you have a trading system that needs to calculate portfolio volatility across millions of price points. Here's what that looks like in pure Python, and then in Rust via PyO3.

use pyo3::prelude::*;

#[pyfunction]
fn calculate_volatility(returns: Vec<f64>) -> PyResult<f64> {
    let mean: f64 = returns.iter().sum::<f64>() / returns.len() as f64;
    let variance: f64 = returns
        .iter()
        .map(|r| (r - mean).powi(2))
        .sum::<f64>() / returns.len() as f64;
    Ok(variance.sqrt())
}

#[pymodule]
fn volatility_calculator(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(calculate_volatility, m)?)?;
    Ok(())
}

Your Python code then imports and uses it like any other module:

from volatility_calculator import calculate_volatility

returns = [0.01, -0.02, 0.015, -0.005, 0.03]
vol = calculate_volatility(returns)
print(f"Volatility: {vol}")

PyO3 handles the marshalling of data between Python and Rust, type conversions, and error handling. The attribute macros like #[pyfunction] and #[pymodule] generate the boilerplate that connects your Rust code to the Python interpreter.

For more complex cases, you can expose entire Rust classes as Python objects. A ride-hailing team might expose a geospatial indexing structure like this:

#[pyclass]
struct SpatialIndex {
    inner: RTree<Point>,
}

#[pymethods]
impl SpatialIndex {
    #[new]
    fn new() -> Self {
        SpatialIndex {
            inner: RTree::new(),
        }
    }

    fn insert(&mut self, lat: f64, lon: f64) {
        self.inner.insert(Point { lat, lon });
    }

    fn nearest_neighbors(&self, lat: f64, lon: f64, count: usize) -> Vec<(f64, f64)> {
        self.inner
            .nearest_neighbor_iter(&Point { lat, lon })
            .take(count)
            .map(|p| (p.lat, p.lon))
            .collect()
    }
}

Now Python code can instantiate and interact with this high-performance spatial index as if it were a native Python object, but it runs with Rust's speed and safety guarantees.

maturin builds wheels

Building Python extensions has historically been painful. You had to compile C code, manage multiple Python versions, create wheels for different architectures—it was error-prone and brittle. maturin automates all of this. It's a build tool specifically designed for PyO3 projects that generates platform-specific Python wheels automatically.

Getting started is straightforward. Create a new project with maturin:

cargo install maturin
maturin new --bindings pyo3 my_fast_module
cd my_fast_module

This scaffolds a Rust project with a Cargo.toml and a pyproject.toml already configured for PyO3. Your Cargo.toml includes:

[package]
name = "my_fast_module"
version = "0.1.0"
edition = "2021"

[dependencies]
pyo3 = { version = "0.20", features = ["extension-module"] }

[lib]
name = "my_fast_module"
crate-type = ["cdylib"]

Once you've written your Rust code with PyO3 annotations, building is one command:

maturin develop

This compiles your Rust code and installs it as a local Python package, mimicking what happens during development. When you're ready to release, you build wheels:

maturin build --release

maturin creates wheels for your current platform and Python version. If you need to distribute across multiple platforms (Linux, macOS, Windows) and Python versions (3.8, 3.9, 3.10, etc.), CI tools like GitHub Actions handle this by running maturin on each platform. Many teams use GitHub Actions workflows to build and publish wheels to PyPI automatically when a new release is tagged.

The wheel is just a compiled binary that pip installs like any other package. Users don't need to have Rust installed or compile anything—they download the pre-built wheel and it works.

Great for hot loops

The sweet spot for PyO3 is computationally intensive loops that are called frequently. Consider a CRM system that needs to score millions of leads against complex engagement rules. The scoring logic is pure computation with no I/O—exactly what Rust excels at.

Here's why PyO3 shines in hot loops:

• Rust's performance. Tight loops run without garbage collection overhead, with better memory layout, and with SIMD optimizations that modern CPUs offer.
• One-time marshalling cost. You pass data once from Python to Rust, compute in Rust, and return the result. The data transfer happens only at the boundary, not inside the loop.
• No GIL contention. Rust code running via PyO3 releases the Python Global Interpreter Lock, allowing other Python threads to run in parallel.

Let's say you're processing a stream of transaction events in an e-commerce platform. Each transaction needs to be validated against fraud rules, enriched with historical data, and categorized. A naive Python implementation loops through events and calls helper functions repeatedly. Rewriting the core validation and enrichment logic in Rust can easily give you 10–100x speedups:

#[pyfunction]
fn validate_and_enrich_transactions(transactions: Vec<Transaction>) -> PyResult<Vec<EnrichedTransaction>> {
    let results = transactions
        .into_iter()
        .filter(|t| is_valid_transaction(t))
        .map(|t| enrich_transaction(t))
        .collect();
    Ok(results)
}

The key is that this runs in a single Rust function call. Python doesn't interpret or manage each iteration—Rust's compiler optimizes the entire loop as one unit.

Many popular tools use it

PyO3 isn't theoretical or niche. It powers production systems at scale. Some well-known examples:

• Pydantic uses Rust validation under the hood for data parsing, making it orders of magnitude faster than pure-Python validators.
• Polars is a DataFrame library written in Rust with Python bindings via PyO3, offering far better performance than pandas for large datasets.
• Cryptography uses PyO3 to expose Rust-based cryptographic operations securely and efficiently.
• uv, the fast Python package installer, is written in Rust and binds to Python via PyO3.

These projects demonstrate that PyO3 isn't just for hobbyists; teams with millions of users rely on it to deliver performance that pure Python simply cannot match. The pattern is consistent: write the performance-critical path in Rust, expose it cleanly via PyO3, and let Python handle orchestration and business logic.

Common mistakes

Mistake 1: Premature optimization. Don't reach for Rust for every slow function. Profile first. Use tools like cProfile or py-spy to identify genuine bottlenecks. Often the problem is algorithmic, not the language.

Mistake 2: Transferring too much data across the boundary. Every call to a PyO3 function has marshalling overhead. If you're calling a Rust function millions of times with small payloads, the overhead can dominate. Instead, batch work or move more logic into Rust.

Mistake 3: Panicking in Rust code without proper error handling. If your Rust code panics, it crashes the Python interpreter. Always use Result and convert errors to PyErr via ?.

Mistake 4: Forgetting to handle the GIL when spawning threads. If your Rust code spawns threads that call back into Python, you must acquire the GIL. PyO3 provides tools for this, but it's easy to forget.

Key takeaways

• PyO3 lets you write performance-critical Python extensions in Rust with minimal boilerplate. Annotate with macros, expose functions and classes, and call them from Python as normal imports.
• maturin automates building and packaging Rust extensions into platform-specific wheels, eliminating the complexity of managing the Python C API and multiple architectures.
• Target PyO3 at genuine hot loops—tight computational loops where Python's overhead is the real problem, not architectural issues or poor algorithms.
• Profile before optimizing. PyO3 adds value when you've identified a real bottleneck and need Rust's speed to solve it.
• Many production systems—Pydantic, Polars, uv—use PyO3 to deliver performance that Python alone cannot achieve, validating its maturity and reliability.