When your service stops responding
You wake up to a page. A critical service that handles order processing is unresponsive—no errors in the logs, the process is still running, but requests pile up in the queue. You SSH into the box, run jps to find the PID, and then pull a thread dump. Within seconds, you spot three threads deadlocked on a database connection pool and two dozen others blocked waiting. Reading a Java thread dump: patterns, pitfalls and fixes is one of the most practical skills you can develop as a backend engineer. Thread dumps are free, instant diagnostics that tell you exactly where your application is stuck and why.
Capturing the thread dump with jcmd
Before you can read a thread dump, you need to capture one. The standard, jdk-native way is jcmd, which requires no additional setup or JVM flags.
jcmd <pid> Thread.printFirst, find your Java process PID:
jps -lThen capture the dump and save it to a file for analysis:
jcmd 12345 Thread.print > dump.txtThe command is non-intrusive and does not stop the JVM. It generates a snapshot of all threads, their states, and their call stacks at that moment. The output is human-readable text that you can grep, diff, and analyze.
Other tools like jstack (older but still common) and profilers like YourKit or JProfiler offer richer UIs, but jcmd is always available and requires no license. In production, it is your fastest friend.
Understanding thread states: RUNNABLE, BLOCKED, WAITING, TIMED_WAITING
Every thread in the dump appears with a state. Learn to read it quickly:
- RUNNABLE: The thread is either executing or ready to execute. High CPU, long flame graphs, or thousands of runnable threads usually point to contention or a CPU-bound loop.
- BLOCKED: The thread is waiting to acquire a monitor lock (synchronized block or method). It is not running, not even waiting for I/O—just stuck on another thread's lock.
- WAITING: The thread called
Object.wait(),Thread.join(), or similar and is waiting for a signal. No timeout. Can hang forever if the signal never comes. - TIMED_WAITING: Same as WAITING, but with a timeout. Common with
Thread.sleep(),Object.wait(long), or socket reads with a timeout set. - NEW: Thread created but not yet started (rare in dumps).
- TERMINATED: Thread finished (also rare).
The state tells you the kind of problem. BLOCKED means a lock contention. WAITING means an expected signal that never arrived. RUNNABLE with a huge stack or many identical ones means a bottleneck or a leak.
Look for BLOCKED threads and their owners
A hung service almost always has BLOCKED threads. Here is what to look for:
"pool-1-thread-3" #15 prio=5 os_prio=0 tid=0x00007f8a2c8a5800 nid=0x4321 waiting for monitor entry [0x00007f8a1e5fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.example.OrderProcessor.processOrder(OrderProcessor.java:42)
- waiting to lock <0x00000000c5a1c2f0> (a java.lang.Object)
at com.example.OrderService.handle(OrderService.java:18)
at java.base/java.lang.Thread.run(Thread.java:834)This thread wants to lock object 0x00000000c5a1c2f0 but cannot. Who owns it? Search the dump for that object address:
"pool-1-thread-5" #17 prio=5 os_prio=0 tid=0x00007f8a2c8a9000 nid=0x4323 runnable [0x00007f8a1e3fd000]
java.lang.Thread.State: RUNNABLE
at com.example.Database.query(Database.java:105)
- locked <0x00000000c5a1c2f0> (a java.lang.Object)
at com.example.OrderProcessor.saveOrder(OrderProcessor.java:55)Thread 5 holds the lock and is stuck in Database.query(). Now the picture is clear: a slow database query holds a synchronized lock, blocking all other threads that need it. The fix might be to remove the synchronization, use a lock-free data structure, or add a timeout.
This pattern—many BLOCKED threads pointing to one owner—is the smoking gun for lock contention.
Spotting a bottleneck: many threads in the same frame
When you see dozens or hundreds of threads with identical or nearly identical stack traces, that is a bottleneck:
"http-nio-8080-exec-1" #19 prio=5 tid=0x... waiting on condition [0x...]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1043)
at java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:416)
at com.example.DatabasePool.getConnection(DatabasePool.java:72)
at com.example.OrderService.fetchData(OrderService.java:25)If you see this same stack 50 times, the database connection pool is exhausted. Threads are queuing up waiting for a connection. The root cause is upstream: either connections are leaking (not being returned), or the application is making more concurrent requests than the pool can handle.
The fix is to either increase the pool size, reduce request volume, or find and close leaked connections.
Another common bottleneck is a synchronous call to a slow external API inside a synchronized block or inside a servlet that fires one thread per request. The call itself is fine, but serializing it under a lock or under limited concurrency multiplies the latency.
Taking multiple dumps and comparing them
A single dump is a snapshot. To confirm a hang or find intermittent issues, take three dumps a few seconds apart:
jcmd 12345 Thread.print > dump1.txt
sleep 3
jcmd 12345 Thread.print > dump2.txt
sleep 3
jcmd 12345 Thread.print > dump3.txtNow compare them:
- Threads in the exact same line across all three dumps? That is a hang. They are stuck, not just slow.
- Different threads BLOCKED each time? Contention is cycling. Threads are making progress but waiting a lot.
- Stack trace changing rapidly? Threads are executing normally. The hang might be upstream (slow backend, network, etc.) or transient.
If a thread is stuck on the same line in all three dumps, you have found the bottleneck. Investigate that line: Is it doing I/O without a timeout? Is it waiting for a lock that is never released? Is it in an infinite loop?
A real example: three dumps show thread 3 always at Socket.read() with no timeout. The thread is talking to a slow microservice that has gone dark. The fix is to add a socket timeout or move the call off the request thread using an async queue.
Common mistakes when reading thread dumps
- Ignoring WAITING and TIMED_WAITING threads: You might assume only BLOCKED threads matter, but a thread waiting forever on a condition variable is equally hung. Check what it is waiting for.
- Trusting the stack trace too much without the state: A thread at
Socket.read()might be RUNNABLE (actively reading, fast network), TIMED_WAITING (slow read, timeout set), or WAITING (no timeout—potential hang). - Confusing "waiting to lock" with "locked": "waiting to lock" means BLOCKED. "locked" (shown in another thread) means that thread owns the lock. Do not conflate them.
- Taking only one dump and assuming it is representative: A single dump can be misleading. Compare three, spaced a few seconds apart, for confidence.
- Overlooking thread pool saturation: If your thread pool size is 20 and you see 20 TIMED_WAITING threads all at the same line, your pool is full and requests are queuing. This is not a thread dump problem; it is a workload problem.
- Missing daemon vs user threads: Daemon threads (marked "daemon") do not prevent JVM shutdown. If your service shuts down unexpectedly, check if critical threads are accidentally marked daemon.
Key takeaways
- Use
jcmd <pid> Thread.printto capture a thread dump—no setup, no overhead, always available in production. - Thread state (RUNNABLE, BLOCKED, WAITING, TIMED_WAITING) tells you the kind of problem immediately.
- BLOCKED threads point to lock contention; search the dump for the lock owner and see what it is doing.
- Many threads with identical stacks indicate a bottleneck—usually a resource (pool, connection, semaphore) or a slow synchronous call.
- Take three dumps spaced seconds apart. Threads in the same line across all three are genuinely stuck; threads changing positions are making progress.
- Compare dumps side-by-side to distinguish between hangs, contention, and transient slowness.
- Always check the full stack trace, not just the top frame, and pay attention to lock addresses when debugging BLOCKED threads.






Comments
Loading comments…