The bulkhead pattern: a practical Design Patterns guide

We've all watched a single slow dependency bring down an entire service. A reporting API that runs in the background shouldn't be able to consume every database connection your main checkout flow needs. The bulkhead pattern: a practical Design Patterns guide teaches us to isolate failure domains so one leak doesn't flood the whole ship.

This pattern comes from naval architecture—compartments in a ship's hull can be sealed off if one fails, preventing total loss. In distributed systems, we apply the same principle: separate resources, thread pools, connection limits, and even deployments to ensure that degradation in one area doesn't cascade through the entire application.

The ship analogy: compartments stop one leak from flooding the whole ship

Imagine a cargo ship with no bulkheads—just one giant hull. A single collision below the waterline floods everything. Now imagine the same ship divided into sealed compartments. A leak in one section can be contained; the others stay dry and the ship stays afloat.

In software, we face the same risk. Without bulkheads, a runaway process, a database deadlock, or a stuck connection can consume shared resources so completely that unrelated features grind to a halt. We've seen this happen: one background job holding all the connection pool, or one slow third-party API consuming every thread in the executor, leaving the user-facing API starving for workers.

The bulkhead pattern says: compartmentalize. Give each critical dependency or workload stream its own pool of resources. If one compartment gets into trouble, the others can still function. The ship stays operational.

This isn't just defensive programming—it's how real-world systems at scale survive. When a payment provider is slow, your search results should still be fast. When a reporting job is backing up, your API should keep serving requests. That's the promise of bulkheads.

Separate pools: different connection pools or thread limits per dependency

The most concrete way to implement bulkheads is through resource pooling. Instead of one shared pool for all database connections or all threads, create separate pools for different workloads or dependencies.

Let's say your e-commerce platform has three main concerns: the checkout flow, product search, and async reporting. Each hits the database, but they have different performance requirements and impact levels. A shared connection pool means the reporting job can starve checkout of connections.

Here's a practical setup with NestJS and TypeORM:

import { DataSource, DataSourceOptions } from 'typeorm';

// Checkout uses a dedicated pool with tight limits
const checkoutDataSource = new DataSource({
  type: 'postgres',
  host: 'db.internal',
  port: 5432,
  database: 'commerce',
  entities: [CheckoutEntity],
  maxConnections: 20,
  maxIdleTime: 30000,
} as DataSourceOptions);

// Search uses its own pool, can afford more connections
const searchDataSource = new DataSource({
  type: 'postgres',
  host: 'db.internal',
  port: 5432,
  database: 'commerce',
  entities: [SearchEntity],
  maxConnections: 50,
  maxIdleTime: 60000,
} as DataSourceOptions);

// Reporting gets a small, isolated pool
const reportingDataSource = new DataSource({
  type: 'postgres',
  host: 'db.internal',
  port: 5432,
  database: 'commerce',
  entities: [ReportingEntity],
  maxConnections: 5,
  maxIdleTime: 120000,
} as DataSourceOptions);

await checkoutDataSource.initialize();
await searchDataSource.initialize();
await reportingDataSource.initialize();

Now checkout and search have their own connections. If reporting runs a slow query and exhausts its small pool, checkout and search are unaffected. Each bulkhead has its own limits.

The same principle applies to thread pools or worker limits. If you're using a queue system like Bull or RabbitMQ, give each queue processor its own concurrency setting:

// Checkout processor: high concurrency, short TTL
checkoutQueue.process(10, async (job) => {
  // Fast, critical path
  return await processCheckout(job.data);
});

// Reporting processor: low concurrency, can wait longer
reportingQueue.process(2, async (job) => {
  // Slower, non-critical
  return await generateReport(job.data);
});

Each queue pulls from the same Redis, but the processor concurrency limits ensure reporting can't starve checkout of worker availability.

Concurrency limits: a slow reporting API can't consume all workers

Beyond database connections, the biggest resource contention point is often the application's thread or worker pool. If you have 100 worker threads and a single slow API dependency locks 80 of them while waiting for responses, you've only got 20 threads left for everything else.

This is where explicit concurrency limits per dependency become critical. Use semaphores or bulkhead libraries to cap how many concurrent requests can hit each external service.

Here's a practical example using a simple semaphore pattern:

class Semaphore {
  private available: number;
  private waitQueue: (() => void)[] = [];

  constructor(maxConcurrent: number) {
    this.available = maxConcurrent;
  }

  async acquire(): Promise {
    if (this.available > 0) {
      this.available--;
      return;
    }

    return new Promise((resolve) => {
      this.waitQueue.push(() => {
        this.available--;
        resolve();
      });
    });
  }

  release(): void {
    const next = this.waitQueue.shift();
    if (next) {
      next();
    } else {
      this.available++;
    }
  }
}

class DependencyClient {
  private checkoutSemaphore = new Semaphore(50);
  private reportingSemaphore = new Semaphore(5);

  async callCheckoutAPI(data: unknown): Promise {
    await this.checkoutSemaphore.acquire();
    try {
      return await fetch('https://checkout-service/process', {
        method: 'POST',
        body: JSON.stringify(data),
      }).then((r) => r.json());
    } finally {
      this.checkoutSemaphore.release();
    }
  }

  async callReportingAPI(data: unknown): Promise {
    await this.reportingSemaphore.acquire();
    try {
      return await fetch('https://reporting-service/generate', {
        method: 'POST',
        body: JSON.stringify(data),
      }).then((r) => r.json());
    } finally {
      this.reportingSemaphore.release();
    }
  }
}

Now checkout requests can use up to 50 concurrent slots at the reporting API, while reporting calls get exactly 5. If the reporting API is slow and all 5 slots fill up, new reporting requests queue locally instead of holding threads. Checkout isn't affected.

Libraries like opossum (circuit breaker) or bull (queue with concurrency control) bake this in, but understanding the mechanics helps you design your own bulkheads.

Separate deployments: critical paths on their own instances

The ultimate bulkhead is deployment isolation. If you're running everything on shared infrastructure—one Kubernetes cluster, one set of processes—you're trusting that resource contention won't happen. That's a risky bet at scale.

For critical paths, consider separate deployments. Your checkout API runs on its own set of instances with no other workloads. Your background reporting jobs run elsewhere. Your search service has its own cluster. This ensures a runaway process or a noisy neighbor in one system can't degrade another.

In practice, this might look like:

• Checkout service: 5 instances, reserved CPU and memory, strict SLA monitoring.
• Search service: 10 instances, can scale based on query load.
• Reporting/background jobs: 2-3 instances, lower priority, can use burst capacity.
• Admin/internal tools: Shared instances, non-critical.

This separation is expensive in infrastructure costs, but it buys you predictability and failure isolation. When search indexing goes haywire, checkout doesn't notice. When reporting runs a long-running aggregation, the API stays responsive.

Even within Kubernetes, this translates to separate Deployments, separate node pools (with resource requests/limits), and sometimes separate clusters for extreme isolation.

Common mistakes

Setting limits too low: A bulkhead with only 2 concurrent slots will introduce queue latency even during normal operation. Size limits based on peak load patterns for each workload, not theoretical minimums.

Sharing circuit breakers across bulkheads: If checkout and reporting both call the same API and share a circuit breaker, a spike in reporting load can trip it and block checkout. Use separate circuit breakers per bulkhead.

Forgetting about cascading calls: If checkout calls payment processing, which calls fraud detection, which calls an external vendor, each hop needs its own concurrency limits. Otherwise you're just moving the bottleneck.

Not monitoring bulkhead saturation: If your reporting semaphore is consistently full and queueing requests, that's a signal to increase capacity or move the workload. Silently queuing failures in the background defeats the purpose.

Key takeaways

• Bulkheads isolate failure domains—compartmentalize resources so one problem doesn't cascade.
• Separate connection pools, thread pools, and semaphores per dependency prevent resource starvation across workloads.
• Explicit concurrency limits act as a circuit breaker before resources are exhausted; requests queue gracefully instead of hanging threads.
• For critical paths, separate deployments provide the strongest isolation guarantee, trading infrastructure cost for predictability.
• Monitor bulkhead saturation and adjust limits as load patterns change; a constantly full bulkhead is a signal to scale or redesign.
• Bulkheads work best alongside other resilience patterns: circuit breakers, retries with backoff, and timeouts.