git bisect: find the bad commit — lessons from production

Introduction

You merge a pull request on Friday afternoon, and by Monday morning, a customer reports a critical bug in production. The culprit? A commit somewhere in the last fifty changes. You could step through git log one by one, or you could use git bisect: find the bad commit — lessons from production. Binary search through history. This is the tool that turns a frustrating needle-in-a-haystack hunt into a systematic, automated investigation that takes minutes instead of hours.

We've all been there: a feature worked yesterday, it's broken today, and the changelog shows twelve commits since the last known good state. Hunting manually is inefficient and error-prone. Git bisect uses binary search—the same algorithm behind finding an element in a sorted array—to isolate the exact commit that introduced the regression. In production environments where minutes of downtime cost real money, knowing how to bisect efficiently can be the difference between a quick fix and a firefighting marathon.

When you need git bisect

Before diving into mechanics, let's be clear about the scenario. You have a working state (a commit, a tag, or a release branch) and a broken state (usually HEAD or a test environment). The problem emerged sometime between them, but you don't know when. The bug could be a runtime error, a performance regression, a data corruption issue, or a silent behavioral change. Git bisect shines when:

• The broken state is reproducible and testable (automated or manual).
• You have a clear "last known good" reference point.
• The number of commits between good and bad is large enough that manual inspection would be tedious.

In one e-commerce platform we worked with, a payment redirect started failing intermittently after a sprint of twenty-three commits across three pull requests. The team had no idea which change broke the OAuth flow. Using bisect, they narrowed it down to a single commit in under ten minutes and discovered a subtle timing change in token refresh logic that only failed under high concurrency—something no code review had caught.

Start — Mark a bad and a good commit

The first step is to tell git where you are: in a bisect session. You mark the current state as bad (broken) and provide a known good state (working). Git uses these bookends to calculate the middle commit to test next.

git bisect start
git bisect bad
git bisect good v1.4.0

Let's walk through this. git bisect start enters bisect mode and resets your HEAD to the initial state. You then run git bisect bad with no argument, which marks the current HEAD as the bad commit. Finally, git bisect good v1.4.0 marks a known good state—in this case, the tag v1.4.0.

The good reference doesn't have to be a tag. It can be a branch name, a commit hash, or a relative reference like HEAD~20. Pick whatever you're confident works correctly. If you're not sure, test it first in a separate shell before starting bisect.

After running these commands, git calculates the midpoint between good and bad and checks out a commit roughly halfway through the history. Your working directory updates automatically, and your shell prompt often changes to indicate you're in a bisect session.

$ git bisect start
$ git bisect bad
$ git bisect good v1.4.0
Bisecting: 45 revisions left to test after this (roughly 6 steps)
[abc1234def5678...] Merge pull request #142: Add caching layer

That output tells you everything you need: forty-five commits remain to be tested, and it expects roughly six iterations (log₂ 45 ≈ 6). Git has checked out a commit in the middle for you to test.

Test and mark — Git checks out commits; you mark them good or bad

Now you're at a checkout point git selected. This is where the real work happens. You test whether the current commit is good or bad. This might mean running your test suite, spinning up a local dev server and manually reproducing the issue, or checking logs in a staging environment.

Let's say you're debugging a data serialization issue in a NestJS backend. You might:

npm test -- --testPathPattern=serialization
# Tests pass or fail

Or for a React frontend:

npm start
# Manually navigate to the affected page and check for the bug

Once you've determined if the current commit is good or bad, mark it:

git bisect good    # if tests pass or the feature works
# or
git bisect bad     # if tests fail or the bug is present

Git then checks out the next candidate commit and repeats. Each time you mark a commit, the search space shrinks by half. If you marked the last one bad, git will now test a commit earlier in history. If you marked it good, it tests later. The process continues until git identifies the exact commit where the bad behavior started.

Here's what a typical session looks like:

$ git bisect good
Bisecting: 22 revisions left to test after this (roughly 5 steps)
[def567890abc123...] Fix TypeError in user reducer

$ npm test -- --testPathPattern=auth
# Tests pass

$ git bisect good
Bisecting: 11 revisions left to test after this (roughly 4 steps)
[abc111222def333...] Update payment gateway integration

$ npm test -- --testPathPattern=payment
# Tests fail

$ git bisect bad
Bisecting: 5 revisions left to test after this (roughly 3 steps)
[xyz444555aaa666...] Refactor API response formatter

# ... continue testing and marking ...

The beauty of this approach is its logarithmic efficiency. With fifty commits, you'll converge in roughly six tests. With five hundred, still only ten. This beats the linear search of checking every commit manually.

Automate — git bisect run npm test runs it for you

For many teams, especially those with solid test coverage, manual testing at each step is unnecessary. If you have a reliable automated test that fails when the bug is present and passes when it's fixed, you can let git bisect run the entire search for you.

The git bisect run command takes a shell command and executes it at each commit. It interprets the exit code: zero means the commit is good, non-zero means it's bad. Git automatically marks and advances until the bad commit is found.

git bisect run npm test

This single command replaces the entire manual loop. Git checks out each candidate commit, runs your test suite, and logs results. For a test suite that takes thirty seconds, bisecting through fifty commits might take five minutes of wall time instead of thirty minutes of human interaction.

In a trading platform we worked on, a bug in position calculation only manifested under specific market conditions. Rather than manually reproducing the scenario at each commit, the team wrote a minimal test that simulated the scenario:

git bisect run npm test -- --testNamePattern="position calculation under gap conditions"

Bisect ran this test automatically, found the culprit in four minutes, and the engineer was able to trace a subtle floating-point rounding change introduced in a refactor.

You can make the run command as specific as you need. Test a single file, a specific test suite, or a custom script that checks your application in a unique way:

git bisect run ./scripts/regression-check.sh
git bisect run npm test -- --testPathPattern=auth
git bisect run python test_performance.py

The only requirement is that the command exits with zero (success) or non-zero (failure). If you want git to skip a commit—because it doesn't compile, for example—exit with code 125, and git will treat it as untestable and try a different commit.

Finish — git bisect reset returns to your branch

Once the bisect is complete, git reports the exact bad commit with its hash and message:

abc1234def5678 is the first bad commit
commit abc1234def5678
Author: Sarah Chen <sarah@company.dev>
Date:   Thu Sep 14 15:23:44 2024 +0200

    Add batch processing queue without semaphore

    This change skips semaphore initialization for batch jobs
    to reduce startup time, but creates a race condition when
    multiple workers process the same job ID.

At this point, you're still in bisect mode, still checked out at that bad commit. You can inspect the code, review the full diff, or check the associated pull request. Once you're satisfied with your findings, reset back to your original branch:

git bisect reset

This checks you back out to the branch or commit where you started the bisect, and your working directory returns to normal. The bisect session is over.

From here, you can file a bug, open a pull request to fix the regression, or cherry-pick the fix into a release branch. The information from bisect becomes part of your investigation summary: "Commit abc1234 introduced the issue by removing the semaphore initialization; fix committed in pull request #203."

Common mistakes

Confusing good and bad. It's easy to mark a commit backwards, especially under stress. Always double-check: is the current state broken (bad) or working (good)? If unsure, test explicitly before marking.

Using an untested good reference. If your "good" commit is itself broken, bisect will never find the real culprit. Always verify your good reference works before starting.

Not handling git bisect skip. If a commit doesn't compile or is untestable for another reason, use git bisect skip instead of marking it bad. This tells git to try a different commit, not to assume that commit is bad.

Running bisect on the wrong branch. Make sure you're on a branch where the bug is present and you have a good reference point. Bisecting the wrong history is a waste of time.

Misinterpreting automated test results. If you use git bisect run, ensure your test reliably detects the bug and doesn't have flaky passes or false negatives. A unreliable test will lead you to the wrong commit.

Key takeaways

• Bisect is binary search for git history. It's exponentially faster than manual hunting, especially across large commit ranges.
• Mark clearly. Start with git bisect start, then git bisect bad (current state) and git bisect good (known working state).
• Test methodically. At each checkout, verify if the commit is good or bad, then mark it. Repeat until convergence.
• Automate when possible. git bisect run lets git drive the entire search if you have a reliable automated test.
• Reset when done. git bisect reset cleans up and returns you to your original branch.
• Document your findings. Once you've found the bad commit, link it to your bug report or fix PR so the team understands the root cause.