Logistic regression for classification, explained simply

Logistic regression for classification, explained simply

After months of chasing shiny models with dozens of hyperparameters, we often return to logistic regression. It's the classification workhorse that predicts probabilities between 0 and 1, works reliably with scaled features, and produces coefficients we can actually explain to stakeholders. Before you reach for gradient boosting or neural networks, logistic regression should always be your baseline.

In this post, we'll walk through why logistic regression remains one of the most practical tools in a classification workflow. We'll cover the mechanics, show you how to interpret the output, and share patterns we've learned from production systems in fintech, e-commerce, and SaaS.

Why probabilities matter more than you think

Logistic regression doesn't predict binary labels directly. Instead, it outputs a probability—a score between 0 and 1 that represents the model's confidence that an instance belongs to the positive class.

This distinction matters in the real world. In fraud detection, you don't want a hard yes/no; you want a risk score so your team can set thresholds based on tolerance. In lead scoring for SaaS, a probability lets you prioritize which prospects your sales team calls first. In medical applications, a probability gives doctors context.

The math behind this is elegant. Logistic regression applies a sigmoid function to a linear combination of features:

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

# Example: predicting churn in a subscription service
X = np.array([
    [5, 2.5],      # tenure (months), support_tickets
    [24, 0.1],
    [3, 8.2],
    [18, 1.0],
])
y = np.array([0, 1, 0, 1])  # churned or not

# Always scale features for logistic regression
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression()
model.fit(X_scaled, y)

# Get probabilities, not just predictions
proba = model.predict_proba(X_scaled)
print(proba[:, 1])  # probability of class 1
# Output: [0.25, 0.92, 0.18, 0.87]

Notice we're getting back four probabilities. The first customer has a 25% churn risk; the second, 92%. You can now make business decisions: maybe customers above 80% risk get a retention call, those below 30% are likely safe.

Scaled features unlock interpretability

Logistic regression's coefficients are only meaningful when features are on the same scale. Without scaling, a feature with a range of 1–100 will dominate one with a range of 0–1, regardless of true importance.

We always use StandardScaler (zero mean, unit variance) before fitting. This step takes seconds but transforms your results from noise into signal.

from sklearn.preprocessing import StandardScaler
import pandas as pd

# Real scenario: predicting conversion in an e-commerce platform
data = pd.DataFrame({
    'session_duration_seconds': [45, 1200, 23, 890],
    'price_in_cents': [599, 15999, 799, 4999],
    'is_mobile': [1, 0, 1, 0],
    'converted': [0, 1, 0, 1],
})

X = data[['session_duration_seconds', 'price_in_cents', 'is_mobile']]
y = data['converted']

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression()
model.fit(X_scaled, y)

# Coefficients after scaling
for feature, coef in zip(X.columns, model.coef_[0]):
    print(f"{feature}: {coef:.3f}")

After scaling, a coefficient of 0.85 for session duration and 1.2 for is_mobile tells a real story: being on mobile has slightly stronger predictive power than session length. Without scaling, you'd be comparing apples to melons.

Coefficients are your explainability layer

This is where logistic regression shines compared to tree ensembles or neural networks. Each coefficient directly tells you the direction and strength of each feature's influence on the probability.

A positive coefficient increases the predicted probability; negative decreases it. The magnitude shows the strength of that relationship. In a regulated environment—credit decisions, medical diagnosis, hiring—this transparency is not just nice to have, it's mandatory.

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler

# Predicting loan approval
X = np.array([
    [35, 85000, 2],      # age, annual_income, credit_score_range
    [28, 45000, 1],
    [55, 150000, 3],
    [42, 95000, 2],
])
y = np.array([1, 0, 1, 1])

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

model = LogisticRegression()
model.fit(X_scaled, y)

print(f"Intercept: {model.intercept_[0]:.3f}")
print(f"Age coefficient: {model.coef_[0][0]:.3f}")
print(f"Income coefficient: {model.coef_[0][1]:.3f}")
print(f"Credit score coefficient: {model.coef_[0][2]:.3f}")

# Example output:
# Intercept: -0.1
# Age coefficient: 0.12
# Income coefficient: 1.8
# Credit score coefficient: 2.3

Now you can explain to a loan applicant: "Higher credit score has the strongest positive impact on approval. Higher income also helps significantly. Age has a modest positive effect." Try that with a neural network.

Always start with logistic regression

We've built systems where teams jumped straight to XGBoost or LightGBM. Six months in, when the model drifts or behaves unexpectedly, they wish they'd started with a baseline they understood.

Logistic regression should be your first classifier on any new dataset. It takes minutes to implement, trains instantly even on millions of rows, and gives you concrete metrics: accuracy, precision, recall, ROC-AUC. Most importantly, it reveals what features actually matter.

We've found that in many domains—churn prediction, fraud detection, lead scoring—logistic regression achieves 85–95% of the performance of more complex models, with a fraction of the maintenance burden.

from sklearn.metrics import roc_auc_score, classification_report
from sklearn.model_selection import cross_val_score

# Establish your baseline
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

model = LogisticRegression(max_iter=1000)
model.fit(X_train_scaled, y_train)

# Evaluate
y_pred_proba = model.predict_proba(X_test_scaled)[:, 1]
auc = roc_auc_score(y_test, y_pred_proba)
print(f"ROC-AUC: {auc:.3f}")

# Cross-validation to check stability
cv_scores = cross_val_score(model, X_train_scaled, y_train, cv=5, scoring='roc_auc')
print(f"CV AUC: {cv_scores.mean():.3f} ± {cv_scores.std():.3f}")

This baseline tells you immediately: can simpler models solve this, or do you need to invest in something heavier? Often, the answer surprises you.

Common mistakes

• Forgetting to scale features. Logistic regression assumes all features contribute via linear combinations. Without scaling, high-magnitude features drown out others. StandardScaler is non-negotiable.
• Using predictions instead of probabilities. predict() gives you 0 or 1 based on a 0.5 threshold, but predict_proba() gives you the rich probability. Always use probabilities for evaluation and business logic.
• Ignoring class imbalance. If 95% of your data is negative class, logistic regression will naturally skew toward predicting 0. Use class_weight='balanced' or adjust your threshold during evaluation.
• Not checking feature distributions. Logistic regression assumes a linear relationship on the log-odds scale. If a feature is bimodal or has extreme outliers, you may need binning or transformation.
• Skipping the baseline model. Teams sometimes add logistic regression late, discovering it beats the complex model they spent weeks tuning. Start here, not there.

Key takeaways

• Logistic regression outputs probabilities between 0 and 1, not hard binary labels. This is a feature, not a limitation.
• Always scale features with StandardScaler before training. Coefficients are only interpretable at the same scale.
• Coefficients directly explain feature importance and direction. This transparency is invaluable in regulated industries and stakeholder discussions.
• Logistic regression is your baseline. It's fast, stable, and often performs as well as far more complex alternatives.
• Use ROC-AUC and cross-validation to measure performance, then compare against fancier models. You'll often find the overhead isn't worth the gain.