Overfitting and underfitting: a practical AI & ML guide

Introduction

We've all shipped a model that worked perfectly in our notebooks but failed in production. The root cause usually boils down to one fundamental problem: overfitting and underfitting: a practical AI & ML guide is what separates models that generalize from those that don't. Understanding when your model is memorizing noise versus when it's too simple to capture signal is the core challenge of machine learning.

In this guide, we'll walk through both problems, show you how to detect them, and give you concrete tactics to fix them. By the end, you'll have a mental model and practical tools to diagnose and fix generalization issues in your own pipelines.

Overfitting — Great training score, poor test score

Overfitting happens when your model learns the training data too well, including its noise and quirks. The result is a model that performs spectacularly on data it's seen before but crumbles on new data.

Picture a fraud detection system trained on transaction data from a single bank branch. The model learns not just the patterns of real fraud, but also the specific quirks of that branch's customers: the lunch hour spikes, the Thursday afternoon rush, the fact that the manager always approves large transfers at 3 PM. When you deploy it to a different branch or time period, it fails because those learned quirks don't apply.

In code, overfitting looks like this:

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np

# Generate synthetic data: y = 2x + noise
np.random.seed(42)
X = np.linspace(0, 10, 50).reshape(-1, 1)
y = 2 * X.ravel() + np.random.normal(0, 2, 50)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Simple model: fits the true relationship
model_simple = LinearRegression()
model_simple.fit(X_train, y_train)
train_score_simple = model_simple.score(X_train, y_train)
test_score_simple = model_simple.score(X_test, y_test)

print(f"Simple model - Train R²: {train_score_simple:.3f}, Test R²: {test_score_simple:.3f}")

# Overfit model: polynomial degree 15, memorizes noise
poly_features = PolynomialFeatures(degree=15)
X_train_poly = poly_features.fit_transform(X_train)
X_test_poly = poly_features.transform(X_test)

model_overfit = LinearRegression()
model_overfit.fit(X_train_poly, y_train)
train_score_overfit = model_overfit.score(X_train_poly, y_train)
test_score_overfit = model_overfit.score(X_test_poly, y_test)

print(f"Overfit model - Train R²: {train_score_overfit:.3f}, Test R²: {test_score_overfit:.3f}")

Running this shows the classic signature of overfitting: the gap between train and test scores widens dramatically. The polynomial model achieves nearly perfect training accuracy but fails on the test set.

Common causes of overfitting:

• Model too complex — Too many parameters relative to training data size.
• Too much training — Neural networks kept training past the point where they learn signal.
• Not enough data — A small dataset gives the model room to memorize.
• No regularization — The model isn't penalized for large weights.
• Feature explosion — Polynomial features, interaction terms, or too many engineered features without constraint.

Underfitting — Poor on both training and test

Underfitting is the opposite problem. Your model is too simple to capture the underlying pattern, so it performs poorly on both training and test data.

Consider a customer churn prediction model. If you only use a single feature—days since signup—and ignore email engagement, in-app activity, support tickets, and billing history, your model will have high bias. It won't learn enough about what actually drives churn, so both training and validation metrics will be mediocre.

Here's what underfitting looks like:

from sklearn.tree import DecisionTreeRegressor

# Using a very shallow decision tree
model_underfit = DecisionTreeRegressor(max_depth=1)
model_underfit.fit(X_train, y_train)
train_score_underfit = model_underfit.score(X_train, y_train)
test_score_underfit = model_underfit.score(X_test, y_test)

print(f"Underfit model - Train R²: {train_score_underfit:.3f}, Test R²: {test_score_underfit:.3f}")

With max_depth=1, a decision tree can only split the data into two regions. It's too simple to model the true relationship, so the training score itself is poor. The test score won't be much worse—because there's little to overfit to—but both are unacceptably low.

Common causes of underfitting:

• Model too simple — Not enough parameters to learn the signal.
• Insufficient training — Stopping too early or using a learning rate that's too high.
• Poor features — Missing key inputs or using features that don't correlate with the target.
• Too much regularization — Penalties so strong they prevent the model from learning.
• Wrong model family — A linear model won't capture non-linear relationships.

Fixes for overfitting

When you detect overfitting, you have several levers to pull:

1. Collect more data

More training data dilutes the effect of noise and reduces the model's ability to memorize. This is the nuclear option and often the most effective, but not always practical. If you can add high-quality labeled data, do it.

2. Use regularization

L1 (Lasso) and L2 (Ridge) regularization penalize large weights, forcing the model to rely on fewer features or smaller coefficients.

from sklearn.linear_model import Ridge, Lasso

# Ridge regression: L2 regularization
ridge = Ridge(alpha=1.0)
ridge.fit(X_train_poly, y_train)
ridge_train = ridge.score(X_train_poly, y_train)
ridge_test = ridge.score(X_test_poly, y_test)

print(f"Ridge - Train R²: {ridge_train:.3f}, Test R²: {ridge_test:.3f}")

# Lasso: L1 regularization, also does feature selection
lasso = Lasso(alpha=0.1)
lasso.fit(X_train_poly, y_train)
lasso_train = lasso.score(X_train_poly, y_train)
lasso_test = lasso.score(X_test_poly, y_test)

print(f"Lasso - Train R²: {lasso_train:.3f}, Test R²: {lasso_test:.3f}")

You can tune the regularization strength (alpha) using cross-validation to find the sweet spot.

3. Use a simpler model

Sometimes the fix is architectural. A shallow decision tree, fewer neurons in a neural network, or a linear model instead of a kernel SVM can reduce the model's capacity to overfit.

4. Early stopping (for iterative models)

For neural networks and boosted trees, monitor validation loss during training and stop when it stops improving.

from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

# Pseudo-code: early stopping in Keras
model = Sequential([
    Dense(64, activation='relu', input_dim=10),
    Dense(32, activation='relu'),
    Dense(1)
])

model.compile(optimizer='adam', loss='mse')

early_stop = EarlyStopping(
    monitor='val_loss',
    patience=5,
    restore_best_weights=True
)

model.fit(
    X_train, y_train,
    validation_split=0.2,
    epochs=100,
    callbacks=[early_stop]
)

5. Dropout and data augmentation

For deep learning, dropout randomly disables neurons during training, forcing the network to learn redundant representations. Data augmentation artificially expands your training set with transformations.

6. Feature selection

Remove noisy or irrelevant features. Use Lasso, permutation importance, or domain knowledge to prune your feature set.

Learning curves — Seeing which problem you have

The most diagnostic tool is a learning curve: plot training and validation scores against training set size. It tells you whether you're overfitting, underfitting, or hitting diminishing returns.

from sklearn.model_selection import learning_curve
from sklearn.ensemble import RandomForestRegressor
import matplotlib.pyplot as plt

model = RandomForestRegressor(n_estimators=10, max_depth=3, random_state=42)

train_sizes, train_scores, val_scores = learning_curve(
    model, X, y,
    cv=5,
    train_sizes=np.linspace(0.1, 1.0, 10),
    scoring='r2'
)

train_mean = np.mean(train_scores, axis=1)
train_std = np.std(train_scores, axis=1)
val_mean = np.mean(val_scores, axis=1)
val_std = np.std(val_scores, axis=1)

plt.figure(figsize=(10, 6))
plt.plot(train_sizes, train_mean, 'o-', label='Training score', color='blue')
plt.fill_between(train_sizes, train_mean - train_std, train_mean + train_std, alpha=0.2, color='blue')
plt.plot(train_sizes, val_mean, 'o-', label='Validation score', color='red')
plt.fill_between(train_sizes, val_mean - val_std, val_mean + val_std, alpha=0.2, color='red')
plt.xlabel('Training set size')
plt.ylabel('Score (R²)')
plt.legend()
plt.grid(True)
plt.show()

Interpreting the curves:

• Both curves low and flat — Underfitting. Increase model complexity, engineer better features, or use a more expressive model.
• Training high, validation low, gap widens — Overfitting. Add regularization, simplify the model, or collect more data.
• Both curves high and converging — Good fit. Your model is learning well and generalizing. More data may still help if the curves haven't plateaued.
• Curves converge but both low — Increase model capacity, add features, or reconsider your feature engineering.

Common mistakes

Here are pitfalls we see teams hit repeatedly:

• Tuning hyperparameters on the test set — This is overfitting your hyperparameters. Always use a separate validation set or cross-validation.
• Using the same data split for both feature engineering and model selection — You'll find spurious patterns. Use a pipeline that keeps train/test separation strict.
• Ignoring class imbalance — A model that predicts the majority class gets high accuracy on imbalanced data but is useless. Use stratified splits and appropriate metrics.
• Not checking learning curves — You're flying blind. Always plot them early. They take 30 seconds and save hours of wasted tuning.
• Overfitting to your validation set — If you iterate too many times on validation metrics, you'll eventually overfit. Use a held-out test set only at the end.
• Assuming more data always helps — Quality matters more than quantity. Garbage data makes overfitting worse.

Key takeaways

• Overfitting is high training accuracy and low test accuracy. Your model memorized noise. Fix it with more data, regularization, simpler models, or early stopping.
• Underfitting is poor performance on both training and test data. Your model is too simple. Increase capacity, engineer better features, or train longer.
• Learning curves are your diagnostic tool. Plot training vs. validation scores as you increase training set size. They tell you which problem you have and whether more data will help.
• Always use cross-validation or a separate validation set during development, and reserve a held-out test set for final evaluation. This keeps your estimates honest.
• The gap between training and test scores is your canary. A large gap suggests overfitting; no gap despite poor test scores suggests underfitting.
• Regularization strength, model depth, and data size are your main tuning knobs. Iterate on them systematically, not randomly.