Feature engineering that improves models for production

Feature engineering that improves models for production

You can throw the fanciest neural network at a problem, but if your features are weak, your model will fail when it meets real traffic. We've seen teams rebuild models multiple times because they focused on architecture instead of data transformation. Feature engineering that improves models for production often matters more than the algorithm itself—better features beat better algorithms, every time.

In this article, we'll walk through the feature engineering techniques that consistently move the needle in production systems. We're not talking about theoretical exercises; these are patterns from building models for e-commerce search ranking, fraud detection, churn prediction, and ride-hailing matching.

Dates: unlocking temporal patterns with day-of-week, month, and recency

Raw timestamps are almost useless to models. A Unix timestamp is just a number; the model has no context. We need to extract the temporal patterns your domain actually exhibits.

Consider a ride-hailing demand model. Surge pricing patterns are completely different on Friday at 10 PM versus Tuesday at 2 AM. A raw timestamp number tells the model nothing; but day-of-week, hour-of-day, and whether it's a holiday—these are features that matter.

from datetime import datetime
import pandas as pd
import numpy as np

def engineer_date_features(df, timestamp_col='created_at'):
    """
    Extract meaningful temporal features from a timestamp column.
    """
    df['date'] = pd.to_datetime(df[timestamp_col])
    
    # Day of week (0=Monday, 6=Sunday)
    df['day_of_week'] = df['date'].dt.dayofweek
    
    # Is it weekend?
    df['is_weekend'] = (df['day_of_week'] >= 5).astype(int)
    
    # Hour of day
    df['hour'] = df['date'].dt.hour
    
    # Is it during business hours? (9 AM - 5 PM)
    df['is_business_hours'] = ((df['hour'] >= 9) & (df['hour'] < 17)).astype(int)
    
    # Month
    df['month'] = df['date'].dt.month
    
    # Day of month (useful for subscription billing cycles)
    df['day_of_month'] = df['date'].dt.day
    
    # Recency: days since the event
    reference_date = pd.Timestamp('2024-01-15')  # Your model training cutoff
    df['days_since_event'] = (reference_date - df['date']).dt.days
    
    # Quarter
    df['quarter'] = df['date'].dt.quarter
    
    return df

# Example usage
df = pd.DataFrame({
    'user_id': [1, 2, 3, 4, 5],
    'created_at': [
        '2023-12-22 14:30:00',  # Friday afternoon
        '2023-12-25 08:15:00',  # Monday (Christmas)
        '2023-12-29 22:45:00',  # Friday night
        '2024-01-08 11:20:00',  # Monday morning
        '2024-01-12 16:50:00',  # Friday afternoon
    ]
})

df = engineer_date_features(df)
print(df[['created_at', 'day_of_week', 'is_weekend', 'hour', 'days_since_event']])

Recency deserves special attention. In churn prediction, user engagement models, and recommendation systems, how fresh an interaction is matters enormously. A purchase from yesterday signals different intent than one from six months ago. Rather than using raw dates, compute days or hours since the last event. This gives your model something it can meaningfully compare across millions of rows.

In fraud detection, we've built models that flag transactions within hours of account creation because that recency signal is predictive—new accounts committing fraud tend to act fast. Without engineering that feature, the raw timestamp is noise.

Ratios and differences between columns

Models are often better at learning ratios than differences because ratios normalize for scale. If you're predicting revenue per user, feeding the model order value divided by order count is more informative than raw sums.

A real example: in e-commerce, we wanted to predict which products would be returned. Raw features like price, weight, and dimensions were weak. But the ratio of product weight to price—a proxy for perceived quality relative to cost—became one of our strongest signals. Heavy, cheap items get returned more often than light, expensive ones (because expectations are misaligned).

def engineer_ratio_features(df):
    """
    Create ratio and derived features from base columns.
    """
    # Avoid division by zero
    df['price_per_unit_weight'] = np.where(
        df['weight'] > 0,
        df['price'] / df['weight'],
        0
    )
    
    # Revenue efficiency
    df['revenue_per_transaction'] = np.where(
        df['num_transactions'] > 0,
        df['total_revenue'] / df['num_transactions'],
        0
    )
    
    # Engagement ratio
    df['views_per_purchase'] = np.where(
        df['num_purchases'] > 0,
        df['page_views'] / df['num_purchases'],
        0
    )
    
    # Difference features
    df['price_above_category_avg'] = df['price'] - df['category_avg_price']
    
    # Margin indicators
    df['margin_pct'] = np.where(
        df['price'] > 0,
        (df['price'] - df['cost']) / df['price'],
        0
    )
    
    # Handle inf and nan
    df = df.replace([np.inf, -np.inf], 0)
    df = df.fillna(0)
    
    return df

Differences also work: how much higher is this product's price compared to the category average? How much longer is the delivery time versus the promised time? These features encode deviation, which is often what models need to learn.

Aggregates per customer or product

Customer-level and product-level aggregates compress behavioral history into dense features. Instead of feeding a model 200 raw events per user, aggregate them into 10-15 summary statistics.

In travel booking systems, we aggregate per user: total bookings, average spend per booking, standard deviation of booking frequency, cancellation rate, preferred airline, etc. These paint a picture of user behavior in a way that a model can actually use. You're essentially summarizing the user's history into their "persona."

def aggregate_user_features(transactions_df):
    """
    Aggregate transaction-level data into user-level features.
    """
    user_agg = transactions_df.groupby('user_id').agg({
        'order_value': ['sum', 'mean', 'std', 'min', 'max', 'count'],
        'days_since_last_purchase': ['mean', 'min'],
        'product_category': lambda x: x.mode()[0] if len(x.mode()) > 0 else 'unknown',
        'is_returning_customer': 'max',
    }).reset_index()
    
    # Flatten column names
    user_agg.columns = ['_'.join(col).strip('_') for col in user_agg.columns.values]
    
    # Calculate additional features
    user_agg['purchase_frequency'] = user_agg['order_value_count'] / (
        (pd.Timestamp.now() - transactions_df.groupby('user_id')['created_at'].min()).dt.days + 1
    )
    
    # Lifetime value
    user_agg['customer_lifetime_value'] = user_agg['order_value_sum']
    
    # Consistency (inverse of coefficient of variation)
    user_agg['purchase_consistency'] = np.where(
        user_agg['order_value_mean'] > 0,
        1 / (user_agg['order_value_std'] / user_agg['order_value_mean'] + 0.01),
        0
    )
    
    return user_agg

def aggregate_product_features(reviews_df):
    """
    Aggregate review-level data into product-level features.
    """
    product_agg = reviews_df.groupby('product_id').agg({
        'rating': ['mean', 'std', 'count'],
        'helpful_votes': ['sum', 'mean'],
        'review_length': ['mean'],
    }).reset_index()
    
    product_agg.columns = ['_'.join(col).strip('_') for col in product_agg.columns.values]
    
    # Quality signal
    product_agg['quality_score'] = (
        product_agg['rating_mean'] * np.log1p(product_agg['rating_count'])
    )
    
    return product_agg

Think per-product too. If you're ranking search results or recommendations, aggregate product-level statistics: average rating, number of reviews, return rate, sell-through velocity, revenue contribution. These summaries let the model reason about product quality and popularity without seeing all raw interactions.

A key principle: choose aggregates that compress information without losing predictive signal. Sum, mean, count, and standard deviation are your friends. Min and max can capture outliers. Percentiles (25th, 50th, 75th) reveal distribution shape without assuming normality.

Domain knowledge creates the best features

The features that move metrics the most come from deep understanding of your problem, not from data science textbooks. Your product managers, customer support team, and data analysts know patterns that show up in raw data but aren't obvious from statistics alone.

In one fraud detection system, the single best feature wasn't statistical—it was whether the billing address matched the shipping address. In a ride-hailing context, whether a user books during their typical booking window (based on their personal history) matters more than generic hour-of-day signals. These insights come from business conversations, not data exploration.

Here's a practical workflow:

1. Ask domain experts: What behavior patterns distinguish good from bad? What red flags do they watch for? What does a "normal" user/product/transaction look like?
2. Translate to features: Can you measure that pattern in data? Is it a threshold, a ratio, a count, a difference?
3. Validate the signal: Plot feature distributions for positive and negative cases. Does the feature separate them?
4. Iterate: Starting features rarely survive to production. Test, measure, refine.

In SaaS churn prediction, our best feature wasn't usage frequency—it was whether the customer had switched from monthly to annual billing and back to monthly. That action signaled cost-consciousness and indecision. We only discovered it by talking to the sales team about why customers left.

Domain knowledge also prevents feature leakage. Leakage happens when a feature contains information that wouldn't be available at prediction time. In fraud detection, never use "was_disputed" as a feature—you're predicting fraud; the dispute label comes after. In churn prediction, never use "final_email_open_date"—if someone's churn is imminent, of course they're not opening emails. Talk to product and engineering about what data exists at the moment you'll make a prediction, and only use that.

Common mistakes

Over-engineering without validation: Creating 500 features "just in case" wastes computation and often hurts model performance through noise. Engineer features intentionally. For each new feature, ask: why should this help? Can I see it in the data?

Forgetting about scale: If you engineer a feature in pandas on a sample, but your production model trains on 100M rows, your aggregations will be slow and memory-intensive. Test at scale early. Use database queries (SQL) or Spark for aggregations on large datasets, not just Python loops.

Leaking future information: Using data that wouldn't be available at prediction time is the most common cause of production model failure. Always ask: at the moment I make a prediction, is this feature value already known? If not, it's leakage.

Ignoring missing values: NaN and zero aren't the same. A missing value might mean "no data yet" (which is informative) or "this feature doesn't apply." Treat them deliberately. Sometimes a separate "is_missing" binary feature is more honest than filling zeros.

Static features: If you engineer features once and never update aggregates, your model decays as the underlying distribution shifts. User behavior changes. Product popularity changes. Rebuild aggregates on a schedule—daily, weekly, or monthly depending on your domain.

Key takeaways

• Extract temporal patterns from raw timestamps: day-of-week, hour, recency. These unlock patterns hidden in time-series behavior.
• Build ratio features to normalize scale; differences capture deviation. Both often outperform raw columns.
• Aggregate per customer, product, or other entities to compress behavioral history into summary statistics that models can learn from efficiently.
• Talk to domain experts—product, support, sales—before engineering features. Their intuition about patterns often beats statistical exploration.
• Validate every feature: does it separate signal from noise? Is it available at prediction time? Test it before production.
• Scale matters: engineer features in the system where they'll live (database, Spark, data warehouse), not just in Python locally.
• Better features beat better algorithms. Spend twice the time on features, half the time tuning hyperparameters, and your model will win.