From notebook to production: what developers should know

We've all been there: a Jupyter notebook with working ML code that's a mix of exploratory cells, hardcoded paths, and functions that somehow made it past the research phase. When it's time to move that experiment into production, the gap between a prototype and a reliable service becomes painfully obvious. From notebook to production: what developers should know means understanding that the code itself is only part of the story—reproducibility, automation, and observability are equally critical to success.

Refactor into modules — Functions with tests replace notebook cells

The notebook format encourages experimentation, but production systems need structure. Each cell becomes a potential maintenance headache because dependencies are implicit, state leaks between cells, and testing is manual. The first step is to extract logic into proper Python modules organized by concern: data loading, feature engineering, model inference, and business logic.

Start by identifying the natural boundaries in your notebook. A series of cells that load and clean data become a data.py module. Cells that compute features become features.py. Model prediction logic becomes predict.py. Each module should have clear input and output contracts.

# features.py
def load_raw_data(path: str) -> pd.DataFrame:
    """Load CSV and validate schema."""
    df = pd.read_csv(path)
    required_cols = {'user_id', 'transaction_amount', 'timestamp'}
    if not required_cols.issubset(df.columns):
        raise ValueError(f"Missing columns: {required_cols - set(df.columns)}")
    return df

def compute_features(df: pd.DataFrame) -> pd.DataFrame:
    """Engineer features from raw transaction data."""
    df['day_of_week'] = pd.to_datetime(df['timestamp']).dt.dayofweek
    df['amount_log'] = np.log1p(df['transaction_amount'])
    return df[['user_id', 'day_of_week', 'amount_log']]

def prepare_for_inference(raw_df: pd.DataFrame) -> np.ndarray:
    """Transform raw input into model-ready features."""
    features_df = compute_features(raw_df)
    return features_df.values

Now add unit tests. A simple test suite catches regressions and documents expected behavior for the next person maintaining the code.

# test_features.py
import pytest
import pandas as pd
from features import load_raw_data, compute_features

def test_load_raw_data_valid():
    df = load_raw_data('fixtures/valid.csv')
    assert len(df) > 0
    assert 'user_id' in df.columns

def test_load_raw_data_missing_columns():
    with pytest.raises(ValueError, match="Missing columns"):
        load_raw_data('fixtures/incomplete.csv')

def test_compute_features_output_shape():
    df = pd.DataFrame({
        'user_id': [1, 2, 3],
        'transaction_amount': [10.0, 20.0, 30.0],
        'timestamp': ['2024-01-01', '2024-01-02', '2024-01-03']
    })
    result = compute_features(df)
    assert result.shape[0] == 3
    assert 'day_of_week' in result.columns

Modular code is testable, reusable, and easier to debug. It also makes the next phase—deployment—straightforward because you have small, focused units to containerize and version.

Pin the environment — Lock files and containers for reproducibility

A notebook that runs perfectly on your laptop often breaks in production because the environment differs: NumPy version 1.24 behaves differently from 1.26, scikit-learn's model serialization format changes between releases, and system libraries matter. Reproducibility requires pinning exact versions and containerizing the entire environment.

Start with a dependency lock file. Don't use requirements.txt with loose version specifiers like pandas>=1.0. Instead, generate a lock file that captures exact transitive dependencies.

# requirements-lock.txt (pip-tools output)
# Generated by pip-compile --resolver=backtracking
numpy==1.24.3
pandas==2.0.2
scikit-learn==1.2.2
joblib==1.2.0
python-dateutil==2.8.2
pytz==2023.3
six==1.16.0

Use pip-tools to generate this from a source file that lists only top-level dependencies:

# requirements.txt (source)
numpy>=1.24,<2.0
pandas>=2.0,<3.0
scikit-learn>=1.2,<2.0

Then commit the lock file. When the next engineer deploys, they run pip install -r requirements-lock.txt and get identical versions.

For production services, Docker ensures the entire environment—OS, system libraries, Python version, and Python packages—is reproducible across machines. A minimal Dockerfile looks like this:

FROM python:3.11-slim

WORKDIR /app

# Copy lock file and install dependencies
COPY requirements-lock.txt .
RUN pip install --no-cache-dir -r requirements-lock.txt

# Copy application code
COPY src/ .

# Health check and entry point
HEALTHCHECK --interval=30s --timeout=10s CMD python -c "import sys; sys.exit(0)"
CMD ["python", "-m", "uvicorn", "api:app", "--host", "0.0.0.0", "--port", "8000"]

Build and push to a registry: docker build -t myml-service:v1.0.3 . Then any environment—dev, staging, production—runs the exact same image. Model files should also be versioned inside the image or fetched from a known artifact store with a pinned hash.

Automate training — A pipeline that runs end-to-end without manual steps

In the notebook, you retrain the model manually: download data, run feature engineering, call model.fit(), evaluate, save to disk. In production, this needs to be automated, scheduled, and logged. Use a pipeline orchestration tool—Airflow, Prefect, or even GitHub Actions for simpler cases—to ensure reproducible, auditable training runs.

A basic pipeline has stages: fetch data, validate schema, compute features, train model, evaluate, and publish. Each stage is a discrete task with clear inputs and outputs.

# train_pipeline.py
import logging
from datetime import datetime
import joblib
import pandas as pd
from features import load_raw_data, compute_features
from model import train_model, evaluate_model

logger = logging.getLogger(__name__)

def run_training_pipeline(data_path: str, model_output_path: str):
    """End-to-end training pipeline with logging and error handling."""
    run_id = datetime.utcnow().isoformat()
    logger.info(f"Starting training pipeline run: {run_id}")
    
    try:
        # Stage 1: Load and validate
        logger.info("Loading raw data...")
        df = load_raw_data(data_path)
        logger.info(f"Loaded {len(df)} rows")
        
        # Stage 2: Feature engineering
        logger.info("Computing features...")
        features = compute_features(df)
        logger.info(f"Features shape: {features.shape}")
        
        # Stage 3: Train
        logger.info("Training model...")
        model = train_model(features, df['target'])
        
        # Stage 4: Evaluate
        logger.info("Evaluating model...")
        metrics = evaluate_model(model, features, df['target'])
        logger.info(f"Metrics: {metrics}")
        
        # Stage 5: Save
        logger.info(f"Saving model to {model_output_path}")
        joblib.dump(model, model_output_path)
        
        logger.info(f"Pipeline run {run_id} completed successfully")
        return metrics
        
    except Exception as e:
        logger.error(f"Pipeline run {run_id} failed: {e}", exc_info=True)
        raise

if __name__ == "__main__":
    run_training_pipeline("data/raw.csv", "models/model-latest.pkl")

Orchestrate this with Airflow or schedule it with cron or a cloud scheduler. Each run should be idempotent—same inputs produce the same outputs—so retraining is safe. Log every run's start time, end time, data shape, and metrics so you can diagnose issues later.

Monitor — Logs, metrics and drift checks from day one

The notebook stopped running, or the model is silently making bad predictions. Without monitoring, you won't know until customers complain. Production ML systems need three types of signals: application logs, performance metrics, and model drift detection.

Application logs track pipeline execution and errors. Use structured logging to make parsing and alerting easier:

# logging configuration
import logging
import json

class JsonFormatter(logging.Formatter):
    def format(self, record):
        log_obj = {
            'timestamp': self.formatTime(record),
            'level': record.levelname,
            'logger': record.name,
            'message': record.getMessage(),
            'run_id': getattr(record, 'run_id', None),
        }
        if record.exc_info:
            log_obj['exception'] = self.formatException(record.exc_info)
        return json.dumps(log_obj)

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger = logging.getLogger('ml_service')
logger.addHandler(handler)

Performance metrics track model behavior. Log predictions, confidence scores, latency, and business outcomes (e.g., did the model's recommendation convert?) so you can correlate performance with code or data changes:

# metrics.py
from prometheus_client import Counter, Histogram, Gauge

prediction_counter = Counter(
    'ml_predictions_total',
    'Total predictions',
    ['model_version', 'outcome']
)

prediction_latency = Histogram(
    'ml_prediction_seconds',
    'Prediction latency',
    buckets=[0.01, 0.05, 0.1, 0.5, 1.0]
)

model_score_gauge = Gauge(
    'ml_latest_validation_score',
    'Latest model validation score'
)

# In your prediction function:
with prediction_latency.time():
    prediction = model.predict(features)

if prediction > threshold:
    prediction_counter.labels(model_version='v1.0.3', outcome='positive').inc()
else:
    prediction_counter.labels(model_version='v1.0.3', outcome='negative').inc()

Drift detection monitors whether the model's inputs or performance degrade over time. Check if feature distributions have shifted significantly, or if validation metrics are declining:

# drift.py
from scipy.stats import ks_2samp

def check_feature_drift(reference_data: pd.DataFrame, current_data: pd.DataFrame, threshold: float = 0.05):
    """Detect distributional shift using Kolmogorov-Smirnov test."""
    drifts = {}
    for col in reference_data.columns:
        statistic, p_value = ks_2samp(reference_data[col], current_data[col])
        drifts[col] = {'statistic': statistic, 'p_value': p_value, 'drifted': p_value < threshold}
    return drifts

Wire these signals into a dashboard (Grafana, DataDog, CloudWatch) and alerting system. Alert on prediction latency spikes, error rates, and detected drift so the team can investigate quickly.

Common mistakes

• Skipping tests because "it's just data science." Tests catch bugs early and give confidence when refactoring. Even simple unit tests add value.
• Using loose version pins. Exact versions make reproduction possible. Loose pins invite "it works on my machine" problems.
• Training models manually outside the pipeline. If someone trains a model in a notebook, no one else knows about it, no logs exist, and it's hard to reproduce. Automate it.
• Forgetting to log predictions. You can't debug what you can't see. Log enough detail to reconstruct what the model saw and predicted.
• Not testing the training pipeline end-to-end before production. Test it locally with a subset of data, in staging, and on a schedule before relying on it for live predictions.

Key takeaways

• Modularize notebook code into testable functions organized by concern; use unit tests to catch regressions.
• Pin exact versions in a lock file and containerize with Docker so any environment runs identically.
• Automate training with a pipeline tool; log every run's inputs, outputs, and metrics for auditability.
• Monitor application logs, prediction metrics, and feature drift from day one using structured logging and a metrics store.
• Test the entire pipeline end-to-end before production, and design for reproducibility at every stage.