Queues with SQS: a complete AWS guide for production teams

Why SQS matters for distributed systems

Message queues are the backbone of resilient, scalable systems. When your services need to communicate asynchronously—when one service doesn't need an immediate response from another—SQS sits between them and absorbs the load. This is queues with SQS: a complete AWS guide for production teams, and whether you're handling payment confirmations, image processing, or order fulfillment, understanding how SQS works will transform how you design system architecture. Let's walk through what makes SQS the default choice for decoupling services and smoothing out demand spikes.

Standard queues: at-least-once delivery and best-effort ordering

SQS Standard queues are the most commonly used queue type in production. They guarantee at-least-once delivery, which means every message will be delivered at least once, but could arrive multiple times. This is crucial: your consumer code must be idempotent. If processing a message twice creates two charges or sends duplicate emails, you have a problem.

Standard queues also offer best-effort ordering, not guaranteed ordering. If order matters—if you need operations to execute in strict sequence—you might need FIFO queues instead. But for most scenarios (payments, analytics, notifications), the throughput benefit of Standard queues outweighs the ordering guarantee.

Here's a typical producer pushing messages to SQS:

import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";

const sqsClient = new SQSClient({ region: "us-east-1" });

async function publishOrder(orderId, items) {
  const params = {
    QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/orders",
    MessageBody: JSON.stringify({
      orderId,
      items,
      timestamp: new Date().toISOString(),
    }),
    MessageAttributes: {
      Priority: {
        StringValue: "high",
        DataType: "String",
      },
    },
  };

  try {
    const result = await sqsClient.send(new SendMessageCommand(params));
    console.log("Message sent. ID:", result.MessageId);
  } catch (error) {
    console.error("Failed to send message:", error);
  }
}

publishOrder("order-12345", [{ sku: "WIDGET", qty: 2 }]);

On the consumer side, you receive messages in batches to reduce API calls:

import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from "@aws-sdk/client-sqs";

const sqsClient = new SQSClient({ region: "us-east-1" });

async function processOrders() {
  const params = {
    QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/orders",
    MaxNumberOfMessages: 10,
    WaitTimeSeconds: 20, // Long polling (covered next)
  };

  const data = await sqsClient.send(new ReceiveMessageCommand(params));

  if (!data.Messages) {
    console.log("No messages");
    return;
  }

  for (const message of data.Messages) {
    try {
      const body = JSON.parse(message.Body);
      console.log("Processing order:", body.orderId);

      // Your business logic here
      await fulfillOrder(body);

      // Delete on success
      await sqsClient.send(
        new DeleteMessageCommand({
          QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/orders",
          ReceiptHandle: message.ReceiptHandle,
        })
      );
    } catch (error) {
      console.error("Processing failed, will retry:", error);
      // Don't delete; visibility timeout will re-expose it
    }
  }
}

processOrders();

The key insight: you only delete a message after you've successfully processed it. If an exception occurs, the message stays in the queue and reappears after the visibility timeout expires.

Visibility timeout: prevent duplicate processing during failures

When a consumer receives a message, SQS hides it for a period called the visibility timeout (default 30 seconds). During this window, other consumers won't see it. If the consumer processes the message and deletes it, great—it's gone. If the consumer crashes or times out, the message becomes visible again after the timeout expires, and another consumer can pick it up.

This is elegant but requires careful tuning. Set the visibility timeout longer than your maximum expected processing time. If your order fulfillment takes 45 seconds on average but spikes to 120 seconds, set the timeout to at least 150 seconds to give yourself headroom.

If you realize mid-processing that you need more time, you can extend the visibility timeout:

import { SQSClient, ChangeMessageVisibilityCommand } from "@aws-sdk/client-sqs";

const sqsClient = new SQSClient({ region: "us-east-1" });

async function processWithTimeout(message, timeoutSeconds = 300) {
  try {
    console.log("Starting processing...");
    // Long operation
    await new Promise(resolve => setTimeout(resolve, 10000));

    // Nearing default timeout? Extend it
    await sqsClient.send(
      new ChangeMessageVisibilityCommand({
        QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/orders",
        ReceiptHandle: message.ReceiptHandle,
        VisibilityTimeout: 600, // Extended to 10 minutes
      })
    );

    console.log("Processing complete");
    return true;
  } catch (error) {
    console.error("Failed:", error);
    return false;
  }
}

A common production pattern: set a moderate visibility timeout (60–120 seconds), monitor CloudWatch metrics for ApproximateAgeOfOldestMessage, and adjust if messages are aging too long in the queue.

Dead-letter queues: handle persistent failures gracefully

Some messages will fail repeatedly. A malformed JSON payload, a downstream service outage, or corrupt data might cause your consumer to throw an exception every time it tries to process a message. You don't want those messages stuck in your main queue forever, poisoning your metrics and blocking other messages.

A dead-letter queue (DLQ) is a separate SQS queue where you send messages after a configurable number of failures. Typically, after 3 failed receive attempts, a message moves to the DLQ where it can be inspected, logged, and handled by a different process.

Set up a DLQ via the AWS Console or code:

import { SQSClient, CreateQueueCommand, SetQueueAttributesCommand } from "@aws-sdk/client-sqs";

const sqsClient = new SQSClient({ region: "us-east-1" });

async function setupQueuesWithDLQ() {
  // Create DLQ first
  const dlqResponse = await sqsClient.send(
    new CreateQueueCommand({
      QueueName: "orders-dlq",
      Attributes: {
        MessageRetentionPeriod: "1209600", // 14 days
      },
    })
  );

  const dlqUrl = dlqResponse.QueueUrl;
  console.log("DLQ created:", dlqUrl);

  // Create main queue with redrive policy
  const mainResponse = await sqsClient.send(
    new CreateQueueCommand({
      QueueName: "orders",
    })
  );

  const mainQueueUrl = mainResponse.QueueUrl;

  // Get DLQ ARN (you'd fetch this in production)
  const dlqArn = "arn:aws:sqs:us-east-1:123456789:orders-dlq";

  await sqsClient.send(
    new SetQueueAttributesCommand({
      QueueUrl: mainQueueUrl,
      Attributes: {
        RedrivePolicy: JSON.stringify({
          deadLetterTargetArn: dlqArn,
          maxReceiveCount: 3, // After 3 failures, move to DLQ
        }),
      },
    })
  );

  console.log("Main queue configured with DLQ redrive policy");
}

In production, you'd have a separate consumer listening to the DLQ, logging failures, alerting engineers, or writing them to a database for later analysis. This prevents message loss and gives you visibility into what went wrong.

Long polling: reduce empty receives and lower costs

By default, when you call ReceiveMessage and there are no messages, SQS returns an empty response immediately. Your consumer then polls again milliseconds later. This creates wasted API calls and costs you money.

Long polling tells SQS to wait up to 20 seconds for a message to arrive before returning. If a message shows up within that window, the consumer gets it immediately. If not, it returns empty after the wait. This dramatically reduces API calls and costs, especially for low-traffic queues.

Enable it by setting WaitTimeSeconds:

const params = {
  QueueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/orders",
  MaxNumberOfMessages: 10,
  WaitTimeSeconds: 20, // Long polling enabled
};

const data = await sqsClient.send(new ReceiveMessageCommand(params));

You can also set it as a queue attribute so it's the default for all consumers:

await sqsClient.send(
  new SetQueueAttributesCommand({
    QueueUrl: queueUrl,
    Attributes: {
      ReceiveMessageWaitTimeSeconds: "20",
    },
  })
);

In a real scenario, long polling reduced API calls by 90% for a low-traffic notification queue, cutting costs by about $15/month. For high-traffic queues, the savings are proportional—a queue receiving a million messages daily benefits significantly.

Common mistakes

• Forgetting idempotency: Assuming at-most-once delivery. Always write consumers that handle duplicate messages safely. Use a deduplication key (order ID, transaction ID) to detect and skip duplicates.
• Visibility timeout too short: Setting it to 30 seconds but your processing takes 60 seconds, causing messages to reappear and be processed multiple times. Monitor and adjust based on actual latencies.
• No DLQ configured: Poison messages get stuck in the main queue. Set up a DLQ with a reasonable maxReceiveCount (typically 3–5) from day one.
• Ignoring long polling: Leaving WaitTimeSeconds at 0 and creating unnecessary API calls. This is easy wins for reducing costs and latency.
• Not monitoring queue depth: CloudWatch's ApproximateNumberOfMessagesVisible tells you if consumers are falling behind. Set up alarms when queues grow unexpectedly.

Key takeaways

• Standard SQS queues provide at-least-once delivery and best-effort ordering—build idempotent consumers.
• Visibility timeout must exceed your maximum processing time; adjust it as your system evolves.
• Configure a dead-letter queue to capture persistent failures; monitor and investigate messages that land there.
• Enable long polling to reduce API calls and costs, especially for lower-traffic queues.
• Use batch operations and monitor CloudWatch metrics to catch processing bottlenecks early.