SQS FIFO queues: a complete AWS guide for production teams

When you've built distributed systems that depend on message ordering—payment pipelines, order fulfillment, trade execution—you quickly discover that AWS Standard SQS doesn't guarantee it. That's where SQS FIFO queues come in: they're the lever you pull when the sequence of operations matters as much as the operations themselves.

I've watched teams burn cycles debugging race conditions that could've been prevented with FIFO from day one. This isn't about being overly cautious; it's about picking the right tool. FIFO queues enforce ordering and deduplication, but they come with trade-offs that matter at scale.

Order per message group ID

The core promise of FIFO is deceptively simple: messages with the same MessageGroupId are processed in the order they were sent. This is fundamentally different from Standard queues, where ordering is best-effort at best.

In a typical e-commerce system I worked on, we had user account operations: deposit funds, place order, refund, withdraw. If these messages arrive out of order, the account balance becomes nonsense. With FIFO, we set MessageGroupId to the user ID. Every message for that user goes through a logical queue of its own, and the consumer processes them sequentially.

Here's what that looks like in practice:

aws sqs send-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789/user-operations.fifo \
  --message-body '{"action":"deposit","amount":100}' \
  --message-group-id "user-12345" \
  --message-deduplication-id "deposit-uuid-v4"

That MessageGroupId is the partition key. SQS internally ensures that only one consumer receives messages from the same group at the same time. This serialization is what gives you ordering, but it also caps your throughput for that group to a single consumer.

I've seen teams misuse this by setting the same MessageGroupId for all messages—essentially creating a single global queue. That defeats the entire benefit of a queue cluster. The art is carving out your group IDs so that:

• Messages that must be ordered together share a group ID (user, account, order, trade, etc.)
• Unrelated messages get different group IDs so they can be processed in parallel
• You don't create bottlenecks by over-serializing

In the ride-hailing platform we built, we used FIFO for ride lifecycle events. Each ride_id was its own group, so thousands of concurrent rides could be processed in parallel—but within each ride, events came out in order: matched → accepted → started → completed.

Deduplication within a 5-minute window

Network failures happen. A producer sends a message, gets a timeout, and retries. Sometimes the message was actually delivered the first time. FIFO queues offer built-in deduplication to handle this scenario.

You provide a MessageDeduplicationId—typically a hash or UUID derived from the message content or a business identifier. Within a 5-minute window, if SQS sees two messages with the same deduplication ID in the same message group, it discards the duplicate.

aws sqs send-message \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789/payments.fifo \
  --message-body '{"order_id":"ord-999","amount":50.00}' \
  --message-group-id "ord-999" \
  --message-deduplication-id "charge-attempt-1"

That 5-minute window is critical. It means deduplication IDs are kept in memory for 300 seconds, then discarded. If your producer retries after 5 minutes, SQS has no record of the original message and will treat it as new. For most transactional workloads—payments, orders, trades—5 minutes is plenty. For long-running batch jobs, you might need to handle idempotency at the application level instead.

I've also seen two deduplication strategies:

1. Content-based: Hash the entire message body. Any identical message is considered a duplicate. Safe but risky if your message includes timestamps or other non-deterministic fields.
2. Idempotency key: Require the producer to include a business-level unique identifier (order ID, transaction ID, trade ID). More explicit and usually safer.

In the travel booking system, we used the second approach: MessageDeduplicationId = booking_confirmation_token. The token was generated client-side and stayed constant across retries. Simple, deterministic, no surprises.

Lower throughput than standard queues

Here's the trade-off nobody likes but everyone has to accept: FIFO queues max out at 3,000 messages per second with batching, or 300 without. Standard queues go much higher. You're paying for ordering with a throughput ceiling.

Why? Because ordering within a message group requires serialization. SQS can't hand messages from the same group to multiple workers—that would break order guarantees. So if you have 100 concurrent messages in the same group, only one worker grabs them at a time. Multiply that across thousands of groups, and you hit the queue-level throughput limit faster than you'd expect.

I designed a CRM system that ingested leads. We had 50,000 leads per minute during peak hours. We initially put all leads in a FIFO queue—disaster. The queue became a bottleneck within minutes. The fix: we sharded leads by region and industry into separate FIFO queues, distributing load across multiple queue resources. That's the pattern when FIFO throughput becomes the constraint.

Before choosing FIFO, calculate your expected throughput per message group:

• If you have 10,000 active groups and 5 messages per second total, you're fine (0.0005 msg/sec per group).
• If you have 5 active groups and 1,000 messages per second, you'll hit the queue limit in seconds.

Test with production-like load. I've seen systems that looked fine in staging but failed hard when real traffic arrived. The deduplication window and ordering guarantees sound cheap until you measure them under contention.

Use only when ordering truly matters

This is the discipline I wish I'd learned earlier. Not every queue needs FIFO. In fact, most don't.

Standard queues are:

• Cheaper (FIFO has a higher per-API-call cost)
• Faster (no ordering overhead)
• More scalable (higher throughput ceilings)
• Sufficient for most async tasks (sending emails, logging, webhooks, image resizing)

Use FIFO only when the correctness of your system depends on message order. Some real examples from my work:

• Payment processing: Deposit, then withdraw. Order matters.
• Ride lifecycle: Matched, accepted, started, completed. Order matters.
• Trade execution: Entry, stop-loss, exit. Order matters absolutely.
• Inventory updates: Create item, then update stock. Order matters.
• Notifications: Send welcome email, then send promo email. Order doesn't matter—these are independent.
• Metrics aggregation: Count pageviews, sum revenue, etc. Order doesn't matter—you're aggregating anyway.
• User creation: Create account, then send verification email. These can happen in parallel safely.

I've lost count of teams who picked FIFO preemptively "just to be safe," then spent months wondering why their queue was slow. The correct question to ask is: If these messages arrived in any order, would my system break? If the answer is no, use Standard.

Common mistakes

Using a single global MessageGroupId. I've seen this more than once. A team sets MessageGroupId = "global" for every message to "guarantee ordering." Congratulations—you've just created a single-threaded queue that can handle 300 messages per second max. Use this pattern only if you genuinely need a global ordering constraint (rare).

Not handling deduplication window expiry. Your producer implements aggressive retries with exponential backoff. After 6 minutes, the first attempt falls outside the 5-minute deduplication window. The retry goes through as a new message. If you're not idempotent at the consumer, you process the same business operation twice. Test this scenario explicitly.

Mixing FIFO and Standard queues for related work. I've seen a CRM where lead creation went to a FIFO queue, but lead enrichment went to a Standard queue. The enrichment sometimes ran before the lead existed in the database. If your workflow spans multiple queues, ensure all queues that need ordering are FIFO, or handle eventual consistency at the application layer.

Not sharding when throughput grows. A prototype FIFO queue worked fine with 10 messages per second. Six months later, you're at 500 messages per second. The queue becomes a bottleneck. Plan for sharding from the start if you expect growth. Use Lambda autoscaling or multi-queue consumers to distribute load across parallel FIFO resources.

Forgetting that FIFO doesn't guarantee processing order, only delivery order. FIFO ensures messages arrive in order. A slow consumer or a crash can still cause out-of-order processing. If you need guaranteed processing order (e.g., at-most-once semantics per message), you need application-level checkpointing and visibility timeouts.

Key takeaways

• FIFO queues order messages per MessageGroupId—use this as your business partition key (user ID, order ID, ride ID).
• Deduplication is automatic within 5 minutes; set a MessageDeduplicationId for idempotent retries, but don't rely on deduplication across the 5-minute boundary.
• FIFO throughput is limited (3,000 msg/sec with batching); measure your load per message group and shard aggressively if you exceed this.
• Pick FIFO only when message order is a correctness requirement, not a preference. For most async tasks (email, logging, analytics), Standard queues are the right choice.
• Test ordering and deduplication scenarios explicitly in staging. Edge cases like retries after the deduplication window and consumer crashes are easy to miss.