scikit-learn tutorial: your first model, explained simply

Why your first model matters

We've all been there: staring at a dataset, feeling the weight of machine learning tutorials that jump straight into deep theory. This scikit-learn tutorial: your first model, explained simply strips away the unnecessary complexity and shows you the exact workflow we use in production every day. You don't need a PhD to train a classifier that actually works—you need discipline, good data practices, and about twenty lines of Python.

The real skill isn't memorising algorithms. It's knowing when to split your data, how to avoid fooling yourself with inflated scores, and why feature quality beats model complexity ninety percent of the time.

Split the data — always hold out a test set

Before you write a single line of model code, you need a test set. This is non-negotiable. We've seen teams burn weeks optimising models that looked great on training data but collapsed in production. The reason? They never held out data the model had never seen.

Think of it this way: if you study using the exact exam questions, your exam score tells you nothing about what you actually know. The same logic applies to machine learning.

Here's the pattern we use across nearly every project:

from sklearn.model_selection import train_test_split

# Assume X is your feature matrix and y is your target
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

What's happening here?

• test_size=0.2 means we hold back 20% of our data for final evaluation. On smaller datasets, use 30%. On datasets with millions of rows, 10% is often enough.
• random_state=42 makes the split reproducible. Set this to any integer you like, but use the same value every time. This matters when you're debugging or showing results to colleagues.
• X_train and y_train go to your model. X_test and y_test stay locked away until you've finished training and tuning.

We typically don't touch X_test or y_test until we're confident in our model architecture. If you peek at test performance while tuning hyperparameters, you're leaking information and will overfit.

Train — get a baseline working immediately

Once your data is split, training is almost anticlimactic. Pick a simple model, fit it, move on. Resist the urge to spend days choosing between gradient boosting variants. Start here:

from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

That's it. Your model is trained. RandomForestClassifier is our go-to starter in nearly every classification scenario because it's robust, handles non-linear patterns, doesn't require feature scaling, and gives reasonable results with minimal tuning. We've shipped dozens of projects that use this or something equally straightforward.

Why RandomForest? It's forgiving. It doesn't require careful feature normalisation. It doesn't fail silently. It's easy to inspect for which features matter. And it's fast enough for most production workloads.

The parameters here are sensible defaults:

• n_estimators=100 means we're growing 100 decision trees and averaging their predictions. More trees = better stability, at the cost of speed.
• random_state=42 ensures reproducibility across runs.

Don't overthink this yet. The goal is a working baseline, not a publication.

Evaluate — score on the test set, never on training data

This is where discipline saves your career. Always evaluate on data the model has never seen. Here's the pattern:

from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# Never use this for real evaluation
train_score = model.score(X_train, y_train)

# This is your real evaluation
test_score = model.score(X_test, y_test)

print(f"Training accuracy: {train_score:.3f}")
print(f"Test accuracy: {test_score:.3f}")

# Deeper inspection
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))

If train_score is much higher than test_score, your model is overfitting. This is normal with tree-based models on complex datasets. If both are terrible, your features don't contain predictive signal, and you need to go back and engineer better ones.

We always print the full classification_report because accuracy alone is misleading. In a fraud detection system, you might have 99.9% accuracy while catching zero frauds. The report shows precision, recall, and F1-score per class, which actually matter.

The confusion_matrix shows you exactly where the model is making mistakes—true positives, false positives, true negatives, false negatives. This is invaluable for understanding what to improve next.

Iterate — improve features before trying fancier models

Once you have a baseline, resist the temptation to jump to XGBoost or neural networks. Spend time on features. Features beat algorithms, full stop. We've seen a 10% improvement in a feature pipeline outperform three weeks of hyperparameter tuning on a fancier model.

Here's the hierarchy we follow:

1. Does the test score meet your business requirement? If yes, ship it. Complexity is a liability.
2. If not, engineer better features. Create interaction terms. Log-transform skewed features. Bin continuous variables. Add domain knowledge.
3. Only then try a different model. And test it on the same train/test split.

For example, in a churn prediction project, we might add:

• User tenure binned into seasons (0–3 months, 3–12 months, etc.)
• Interaction terms like purchase_frequency × account_age
• Rolling averages of recent activity
• Flags for anomalous behaviour

Each of these required business understanding, not deeper learning. After one or two feature iterations, we typically see significant gains. Only then do we consider a gradient boosting model.

A complete, minimal example

Let's tie this together with real code:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, classification_report

# Load data
iris = load_iris()
X, y = iris.data, iris.target

# 1. Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# 2. Train
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# 3. Evaluate
train_acc = model.score(X_train, y_train)
test_acc = model.score(X_test, y_test)

print(f"Training accuracy: {train_acc:.3f}")
print(f"Test accuracy: {test_acc:.3f}")

# Detailed evaluation
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred, target_names=iris.target_names))

# Feature importance (step toward iteration)
for i, importance in enumerate(model.feature_importances_):
    print(f"{iris.feature_names[i]}: {importance:.3f}")

Run this on your machine. You'll have a trained classifier, test scores, and a ranked list of which features drive predictions. This is your foundation.

Common mistakes

Evaluating on the training set: The most common mistake. Your train score will always look better. It's meaningless for real-world performance.

Tuning hyperparameters on test data: If you adjust n_estimators or max_depth while watching test scores, you're overfitting to the test set and need a validation set too. For now, split into train (70%), validation (15%), test (15%) if you're doing heavy tuning.

Starting with a complex model: XGBoost, neural networks, and ensemble stacking look impressive. They're almost never necessary for your first pass. Start simple, measure, then upgrade only if needed.

Ignoring class imbalance: In binary classification, if 95% of your labels are class 0, a model that predicts 0 for everything gets 95% accuracy. Always check the label distribution. Use stratify=y in train_test_split and consider F1-score, not accuracy.

Not documenting your random_state: It seems small, but this one number determines whether your colleague can reproduce your results tomorrow. Always set it, always document why.

Key takeaways

• Split first, train second. Hold out test data before you touch your model. Reproducibly, using random_state.
• RandomForest is your friend. It's a robust baseline that handles most scenarios without fussy preprocessing.
• Test scores reveal truth; train scores are lies. Always evaluate on held-out data, and always print detailed metrics like precision and recall, not just accuracy.
• Features before models. Spend your time engineering better features. A simple model on great data beats a complex model on weak data.
• Iterate deliberately. Improve features, measure impact, only then try a different algorithm. This workflow scales from a startup prototype to a billion-row production system.