Introduction
We've all shipped concurrent code that works perfectly on our laptop but mysteriously crashes in production under load. Race conditions and the race detector: a practical guide will help you catch these subtle bugs before they reach your users. Go gives us an exceptional tool—the race detector—that surfaces data races early in development, and knowing how to use it effectively separates reliable systems from fragile ones.
Understanding go test -race
The race detector is a runtime analysis tool built into the Go toolchain. Running go test -race instruments your code to detect concurrent reads and writes to the same memory location that aren't properly synchronized. It adds minimal overhead during testing—typically 5-10x slowdown—making it practical to run in CI pipelines.
When a race is detected, the race detector prints a detailed report showing the goroutine IDs, function names, and file locations of the conflicting operations. This specificity is invaluable for debugging. Here's what it looks like in practice:
package main
import (
"fmt"
"sync"
)
func main() {
var counter int
var wg sync.WaitGroup
for i := 0; i < 1000; i++ {
wg.Add(1)
go func() {
defer wg.Done()
counter++
}()
}
wg.Wait()
fmt.Println(counter)
}
Running this with go test -race will catch the unsynchronized increment immediately. Without the race detector, the test might pass, but you'd see inconsistent results under concurrent load. The tool does the detective work for us—it instruments memory access patterns and flags unsynchronized operations between goroutines.
Shared memory without sync is a race
At its core, a race condition occurs when two or more goroutines access the same memory location concurrently, and at least one of them is writing. Without synchronization, the outcome becomes unpredictable because the CPU's memory model doesn't guarantee ordering or visibility across cores.
Consider a typical e-commerce scenario: tracking inventory updates from multiple order-processing goroutines.
package inventory
type Stock struct {
Available int
}
func (s *Stock) Decrement(quantity int) bool {
if s.Available >= quantity {
s.Available -= quantity
return true
}
return false
}
// Called concurrently from multiple goroutines
func ProcessOrders(stock *Stock, orders []Order) {
for _, order := range orders {
go func(o Order) {
stock.Decrement(o.Quantity)
}(order)
}
}
Multiple goroutines calling Decrement concurrently creates a race. Goroutine A might read the value 100, decide it's sufficient, then context-switch. Goroutine B reads 100, makes its update, writes back 90. Goroutine A resumes and writes 99, losing Goroutine B's decrement. The end result: your inventory count is wrong, potentially overselling products.
The race detector catches this exact pattern because it sees concurrent reads and writes to the same field without synchronization primitives. This visibility is crucial—data races aren't always obvious in code review, but the detector never misses them.
Fix with mutexes or channels
Go provides two main approaches to synchronize shared memory: mutexes for protecting state, and channels for communication-based synchronization. Choose based on your access pattern and code clarity.
Mutex approach
A mutex (mutual exclusion lock) ensures only one goroutine accesses a critical section at a time. For our inventory example:
package inventory
import "sync"
type Stock struct {
mu sync.Mutex
Available int
}
func (s *Stock) Decrement(quantity int) bool {
s.mu.Lock()
defer s.mu.Unlock()
if s.Available >= quantity {
s.Available -= quantity
return true
}
return false
}
The defer s.mu.Unlock() pattern ensures the lock releases even if the function panics. Always keep the critical section small—lock only the code that accesses shared data. Long-held locks reduce concurrency and introduce deadlock risks.
For read-heavy workloads, consider sync.RWMutex, which allows multiple concurrent readers but exclusive writers:
type Stock struct {
mu sync.RWMutex
Available int
}
func (s *Stock) GetAvailable() int {
s.mu.RLock()
defer s.mu.RUnlock()
return s.Available
}
func (s *Stock) Decrement(quantity int) bool {
s.mu.Lock()
defer s.mu.Unlock()
if s.Available >= quantity {
s.Available -= quantity
return true
}
return false
}
Channel approach
Channels enforce synchronization through communication. Instead of protecting shared state, you pass data through a channel, ensuring only one goroutine owns the data at a time. This is often safer and more idiomatic:
type StockManager struct {
commands chan command
}
type command struct {
op string
delta int
reply chan bool
}
func NewStockManager(initial int) *StockManager {
sm := &StockManager{
commands: make(chan command),
}
go sm.run(initial)
return sm
}
func (sm *StockManager) run(available int) {
for cmd := range sm.commands {
switch cmd.op {
case "decrement":
if available >= cmd.delta {
available -= cmd.delta
cmd.reply <- true
} else {
cmd.reply <- false
}
}
}
}
func (sm *StockManager) Decrement(quantity int) bool {
reply := make(chan bool)
sm.commands <- command{op: "decrement", delta: quantity, reply: reply}
return <-reply
}
This approach guarantees that only the run goroutine touches the available variable—no race possible. The downside is slightly more complexity and latency from channel operations, but you gain stronger guarantees about data ownership.
Run race tests in CI
The race detector is only useful if you actually run it. Make it part of your CI pipeline—it should fail the build if any races are detected. Here's a typical CI setup:
# In your Makefile or CI configuration
test-race:
go test -race ./...
test-race-verbose:
go test -race -v ./...
For GitHub Actions, add a workflow step:
- name: Run tests with race detector
run: go test -race -timeout 10m ./...
The race detector significantly slows tests, so consider running it separately from your fast unit test suite. Some teams run full race detection only on the main branch or before releases, while running fast tests on every PR. Others include race tests in full CI but exclude them from local development to keep iteration fast.
Set a reasonable timeout—the detector's instrumentation can make tests run longer than normal. We typically add -timeout 10m to avoid flaky timeouts in CI.
Common mistakes
- Forgetting to defer unlock: If you forget
defer, an early return or panic will leave the lock held, potentially deadlocking the system. Always usedeferwith mutex operations. - Holding locks across network calls: Don't lock before making HTTP requests or database queries. The lock duration explodes, blocking other goroutines unnecessarily. Fetch data, release the lock, then update.
- Not running race tests locally: Many teams only discover races in production or late-stage testing. Run
go test -racelocally before pushing; it's fast enough for regular use. - Mixing synchronization approaches: Use either mutexes or channels for a given piece of state, not both. Mixing them creates confusion and often reintroduces races.
- Assuming go test -race is complete: The race detector requires actual concurrent execution to detect races. Tests that don't trigger the race condition won't catch it. Write tests that exercise concurrent code paths.
- Protecting the wrong granularity: Locking an entire struct when only one field needs protection is wasteful. Lock at the field or operation level where practical.
Key takeaways
- Use
go test -raceregularly in development and always in CI to catch data races before production. - Data races occur when goroutines access the same memory concurrently without synchronization—the race detector reliably finds them.
- Fix races with mutexes for shared state or channels for ownership-based concurrency, choosing based on your access patterns.
- Keep critical sections small, always use
deferwith locks, and avoid holding locks across I/O operations. - Integrate race detection into CI pipelines and make test failures from races non-negotiable before merging code.
Comments
Loading comments…