Background jobs with BullMQ: what developers should know

Background jobs with BullMQ: what developers should know. Move slow work out of the request cycle.

Imagine a user signs up for your SaaS platform at 11 PM on a Friday, and their welcome email takes eight seconds to send because your SMTP server is slow. That delay blocks their login response, frustrates them, and sets a poor first impression. This is where background jobs with BullMQ come in: they let you move slow work out of the request cycle so users get instant feedback while critical tasks run safely in the background.

Why queues matter: the cost of synchronous work

Every request that waits for an email, export, or webhook call is a request that ties up a Node.js worker thread. In production, this compounds quickly. A user initiates a CSV export. Your API calls your database, generates 50,000 rows, writes to disk, and uploads to S3—all before sending a 200 response. Meanwhile, ten other users are blocked waiting for their own requests to finish.

Queues decouple producers from consumers. The producer (your API endpoint) adds a job and returns immediately. A separate processor service handles the work asynchronously, with retries, exponential backoff, and failure tracking built in. You gain:

• Faster API responses and better perceived performance
• Natural resilience: if a processor crashes, queued jobs survive in Redis
• Built-in retry logic with strategies like exponential backoff
• Observability: see which jobs failed, why, and how many times they've been retried
• Scalability: spin up extra processor instances without changing your API code

In real-world e-commerce systems, welcome emails, order confirmations, shipping notifications, and webhook deliveries all live in queues. In CRM platforms, bulk SMS sends and contact sync operations run as background jobs. Trading systems use them for settlement reports. The pattern is universal.

Producer: adding a job from any service

Installing BullMQ starts with Redis. Ensure Redis is running (local development or managed service), then add the package:

npm install bullmq redis

Next, create a queue configuration file. This typically lives in a shared module so any service can inject it:

// queues/email.queue.ts
import { Injectable } from '@nestjs/common';
import { Queue } from 'bullmq';
import Redis from 'ioredis';

const connection = new Redis({
  host: 'localhost',
  port: 6379,
});

@Injectable()
export class EmailQueue {
  private queue: Queue;

  constructor() {
    this.queue = new Queue('email', { connection });
  }

  async addWelcomeEmail(userId: string, email: string) {
    await this.queue.add(
      'welcome',
      { userId, email },
      {
        attempts: 5,
        backoff: {
          type: 'exponential',
          delay: 2000,
        },
      }
    );
  }

  async addOrderConfirmation(orderId: string, userId: string) {
    await this.queue.add(
      'order-confirmation',
      { orderId, userId },
      {
        attempts: 3,
        backoff: { type: 'exponential', delay: 1000 },
      }
    );
  }

  getQueue(): Queue {
    return this.queue;
  }
}

The code above creates an email queue on Redis. The addWelcomeEmail method accepts payload data and retry options. Notice the attempts: 5 and exponential backoff: if sending fails, BullMQ will retry up to 5 times with increasing delays (2s, 4s, 8s, 16s, 32s). This handles transient failures gracefully.

Now inject the queue into any service where you need to trigger email sends:

// auth/auth.service.ts
import { Injectable } from '@nestjs/common';
import { EmailQueue } from '../queues/email.queue';

@Injectable()
export class AuthService {
  constructor(private emailQueue: EmailQueue) {}

  async signup(email: string, password: string) {
    const userId = await this.createUser(email, password);
    
    // Return immediately
    await this.emailQueue.addWelcomeEmail(userId, email);
    
    return { userId, email };
  }

  private async createUser(email: string, password: string): Promise {
    // Database write
    return 'user_123';
  }
}

The signup endpoint now returns in milliseconds. The email send happens in parallel, on a separate worker. If the email service is down or slow, the user's signup flow is unaffected.

Processor: handling jobs with decorators

A processor is a NestJS class that listens to queue events and executes jobs. BullMQ provides a @Processor decorator to register it:

// processors/email.processor.ts
import { Injectable } from '@nestjs/common';
import { Processor, Process } from '@nestjs/bull';
import { Job } from 'bullmq';
import { MailerService } from '../mailer/mailer.service';
import { UserService } from '../user/user.service';

@Processor('email')
@Injectable()
export class EmailProcessor {
  constructor(
    private mailerService: MailerService,
    private userService: UserService,
  ) {}

  @Process('welcome')
  async handleWelcomeEmail(job: Job) {
    const { userId, email } = job.data;
    
    try {
      const user = await this.userService.findById(userId);
      await this.mailerService.send({
        to: email,
        subject: 'Welcome to our platform',
        template: 'welcome',
        context: { name: user.name },
      });
    } catch (error) {
      console.error(`Email job ${job.id} failed:`, error.message);
      throw error; // Triggers retry
    }
  }

  @Process('order-confirmation')
  async handleOrderConfirmation(job: Job) {
    const { orderId, userId } = job.data;
    
    const order = await this.userService.getOrder(orderId);
    await this.mailerService.send({
      to: order.email,
      subject: `Order ${orderId} confirmed`,
      template: 'order-confirmation',
      context: { order },
    });
  }
}

Register the processor in your module:

// email/email.module.ts
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
import { EmailProcessor } from './processors/email.processor';
import { EmailQueue } from './queues/email.queue';

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'email',
      connection: { host: 'localhost', port: 6379 },
    }),
  ],
  providers: [EmailProcessor, EmailQueue],
  exports: [EmailQueue],
})
export class EmailModule {}

When a job enters the queue, BullMQ assigns it a unique ID and stores it in Redis. The processor polls the queue, retrieves jobs, and executes the handler. If the handler throws an error, the job is retried according to your backoff strategy. If it succeeds, the job is marked complete and removed from the queue (or moved to a completion log, depending on your settings).

Observability: Bull Board dashboard

In production, you need visibility into your queue health. Which jobs failed? How many times has a job been retried? Is a processor stuck or crashed? Bull Board answers these questions with a web dashboard.

Install and configure it:

npm install @bull-board/api @bull-board/express

// main.ts
import { createBullBoard } from '@bull-board/api';
import { ExpressAdapter } from '@bull-board/express';
import { BullAdapter } from '@bull-board/api/bullAdapter';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  // Import your queues
  const emailQueue = app.get(EmailQueue).getQueue();
  const exportQueue = app.get(ExportQueue).getQueue();

  const serverAdapter = new ExpressAdapter();
  serverAdapter.setBasePath('/admin/queues');

  createBullBoard({
    queues: [new BullAdapter(emailQueue), new BullAdapter(exportQueue)],
    serverAdapter,
  });

  app.use('/admin/queues', serverAdapter.getRouter());

  await app.listen(3000);
}

bootstrap();

Visit http://localhost:3000/admin/queues and you'll see a dashboard showing:

• Active jobs currently being processed
• Waiting jobs in the queue
• Completed jobs (if retention is configured)
• Failed jobs with stack traces and retry counts
• The option to retry, delete, or promote jobs manually

This observability is critical. If a job consistently fails, you can see the error immediately and fix it. If a processor crashes, you'll notice waiting jobs piling up and can restart the worker. In staging and production, many teams expose Bull Board behind authentication on a private admin route.

Common mistakes

Mistake 1: Not configuring retry logic. Jobs that fail without backoff retry immediately and exhaust quickly. Always set attempts and a backoff strategy. Exponential backoff (2s, 4s, 8s…) is usually best for transient failures like network timeouts.

Mistake 2: Ignoring job payload size. Redis stores job data in memory. Passing entire database records or large objects to job payloads bloats Redis. Instead, pass IDs or minimal metadata, then fetch full data inside the processor.

Mistake 3: Running processors and producers in the same process during development. It works, but masks real issues. In production, you'll run separate processor instances. Test both locally: one instance for your API, another for the processor.

Mistake 4: Not handling processor crashes gracefully. If a processor dies mid-job, that job is stuck. Use process managers (PM2, Kubernetes) to restart processors automatically. Configure a timeout on job handlers so stuck processes don't block forever.

Mistake 5: Forgetting about Redis persistence. If Redis restarts and has no persistence, your queue data disappears. Enable RDB snapshots or AOF (append-only file) in production Redis so jobs survive crashes.

Key takeaways

• Background jobs decouple producers from consumers, letting APIs respond fast while slow work runs asynchronously in the background.
• BullMQ provides retry logic, backoff strategies, and failure tracking out of the box—use them to handle transient failures automatically.
• Producers add jobs to queues via simple service methods; processors handle them via @Processor and @Process decorators.
• Bull Board gives you a dashboard to monitor queue health, failed jobs, and retry counts—essential for production debugging.
• Keep job payloads small (pass IDs, not full objects), run processors separately from your API, and configure retry strategies appropriate to your use case.
• Test queue behavior locally before deploying; many queue-related bugs only surface under load or when processors crash.