Precision vs recall explained: a practical AI & ML guide

You've trained a model, run it against your test set, and got 94% accuracy. Your team celebrates. Two weeks later, fraud losses spike because your fraud detector missed 40% of fraudulent transactions. This is the reality of imbalanced data—accuracy is a misleading metric that hides critical performance gaps. Understanding precision vs recall explained: a practical AI & ML guide is essential for building systems that work in production, where the cost of different types of errors varies wildly.

Why accuracy alone isn't enough

Imagine a fraud detection system trained on transaction data where 99.5% of transactions are legitimate and 0.5% are fraudulent. A naive model that predicts "legitimate" for every transaction achieves 99.5% accuracy without detecting a single fraud case. This is why we need better metrics.

When classes are imbalanced—which they are in most real-world problems—accuracy becomes almost useless. We need metrics that tell us what's actually happening with the minority class, the one we typically care about. That's where precision and recall come in.

Precision — Of everything predicted positive, how much was right?

Precision answers a straightforward question: when our model says something is in the positive class, how often is it correct? It's the ratio of correct positive predictions to all positive predictions.

Precision = TP / (TP + FP)

where:
  TP = True Positives (correct positive predictions)
  FP = False Positives (incorrect positive predictions)

Let's say your spam filter classifies 100 emails as spam. Of those 100, 85 are actually spam and 15 are legitimate emails incorrectly flagged. Your precision is 85/100 = 0.85 or 85%.

High precision means when your model makes a positive prediction, you can trust it. Low precision means your positive predictions include a lot of noise—false alarms that frustrate users or create extra work for your team.

In a spam filter, false positives are annoying but tolerable. A user might miss a legitimate email in their spam folder. But consider a medical imaging system where positive predictions trigger biopsies. A 60% precision means 40% of patients you send for biopsy don't actually have the disease. That's expensive, stressful, and wasteful. For such use cases, we often prioritize high precision.

You can improve precision by raising your confidence threshold for making positive predictions. Instead of flagging emails with a 0.3 spam probability, you only flag those with 0.7+ probability. Fewer false alarms, but you'll miss some actual spam.

Recall — Of all real positives, how many did we catch?

Recall answers a different question: of all the actual positive cases in the data, how many did our model identify? It's also called sensitivity or the true positive rate.

Recall = TP / (TP + FN)

where:
  TP = True Positives (correct positive predictions)
  FN = False Negatives (positive cases we missed)

Back to the spam filter: imagine there are 1,000 actual spam emails in your inbox that week. Your filter catches 850 of them but misses 150. Your recall is 850/1000 = 0.85 or 85%.

High recall means you're catching most of the positive cases. You're not missing many. Low recall means you're letting a lot of positive cases slip through undetected.

In fraud detection, a 60% recall is dangerous. It means 40% of actual fraudulent transactions make it through your system. For a payment processor handling millions of dollars daily, that's substantial financial loss. In medical screening for a serious disease, 60% recall is unacceptable—you're missing 40% of people who need treatment.

You can improve recall by lowering your confidence threshold. Flag emails with just 0.3 spam probability instead of 0.7. You'll catch more spam, but you'll also incorrectly flag more legitimate emails.

The trade-off — Raising the threshold increases precision and lowers recall

This is the central tension in classification: precision and recall exist in inverse relationship for a given model. As you adjust the decision threshold, you move along a curve. Raise it, and precision goes up while recall goes down. Lower it, and recall improves while precision suffers.

Consider a fraud detection model that outputs probabilities. You're currently using a 0.5 threshold: anything above 0.5 is flagged as fraud.

This trade-off is visualized in the Receiver Operating Characteristic (ROC) curve and the Precision-Recall (PR) curve. The ROC curve plots True Positive Rate (recall) against False Positive Rate. The PR curve plots precision against recall. A model with a curve closer to the top-right corner is better—higher recall and higher precision at the same threshold.

You don't choose one threshold and declare victory. You choose a threshold based on your use case and the relative costs of false positives versus false negatives.

Which matters more — Depends on the cost of mistakes

The decision between precision and recall ultimately depends on business logic: what's the cost of each type of mistake?

Fraud detection — Prioritize recall

In payment processing or credit card transactions, a missed fraud case (false negative) costs real money directly—chargeback liability, investigation costs, potential regulatory fines, and customer trust damage. A false positive (flagging a legitimate transaction as fraud) is annoying but usually reversible: the customer calls, confirms the purchase, and it goes through.

The math is clear: false negatives are more costly. You want high recall, even if it means flagging some legitimate transactions. A 70% precision with 95% recall is acceptable in fraud. A 95% precision with 40% recall leaves your system vulnerable.

Spam filtering — Prioritize precision

For email spam filters, a false positive (legitimate email marked spam) might cause users to miss important messages. A false negative (spam reaching the inbox) is annoying but won't break the product—users can delete it or mark it spam next time.

Missing a legitimate email is worse than letting spam through. Your email system should aim for high precision even if recall suffers. 95% precision with 70% recall is preferable to 70% precision with 95% recall.

Medical screening — Prioritize recall

A screening test for cancer or heart disease should catch nearly all true cases, even if it generates some false alarms that require follow-up testing. The cost of a false negative—missing someone who actually has the disease—can be life-threatening. False positives cost money and anxiety but are manageable.

Medical screening demands high recall, typically 95%+. You'll tolerate lower precision because the consequences of missing actual disease are severe.

Working with precision and recall in practice

Here's how you might evaluate a model across both metrics using a common ML library:

from sklearn.metrics import precision_score, recall_score, f1_score, confusion_matrix

# Your predictions and ground truth
y_true = [1, 0, 1, 1, 0, 1, 0, 0, 1, 1]
y_pred = [1, 0, 1, 0, 0, 1, 0, 1, 1, 1]

precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"F1 Score: {f1:.2f}")

# Confusion matrix for detailed breakdown
cm = confusion_matrix(y_true, y_pred)
print(cm)
# [[TN, FP],
#  [FN, TP]]

The F1 score is the harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). It's useful when you want a single metric that respects both dimensions, but only if the costs of false positives and false negatives are similar. If they're not, don't rely on F1 alone.

For imbalanced datasets, the PR curve (Precision-Recall curve) is more informative than the ROC curve because it focuses on the minority class performance without being distracted by true negatives.

Common mistakes

• Using accuracy as the primary metric on imbalanced data: On a 95% negative class dataset, a dumb baseline that predicts negative for everything hits 95% accuracy while catching zero positives. Always check precision and recall.
• Not setting the threshold based on business requirements: The default 0.5 threshold is arbitrary. Adjust it based on your use case's cost structure.
• Confusing precision with positive predictive value: They're the same thing mathematically, but precision is more commonly used in ML discourse.
• Ignoring the threshold trade-off: A model that "performs well" at 0.5 threshold might perform poorly at 0.7. Document which threshold you're using.
• Comparing precision/recall across different datasets without context: These metrics depend on class distribution. A 50% recall on a balanced dataset might represent different model behavior than 50% recall on a highly imbalanced dataset.

Key takeaways

• Accuracy is misleading on imbalanced data. Use precision and recall to understand real model performance.
• Precision measures false alarm rate; recall measures missed cases.
• Precision and recall trade off: lowering the threshold improves recall at the cost of precision.
• Choose your threshold based on business costs, not arbitrary metrics.
• Fraud detection and medical screening prioritize recall; spam filters prioritize precision.
• Use F1 score only when false positive and false negative costs are balanced.
• PR curves are more informative than ROC curves for imbalanced classification problems.