Idempotency keys: a Design Patterns guide for developers

Introduction

Network failures happen. A payment gateway times out mid-transaction, your SaaS backend crashes during an API call, or a retry library aggressively re-sends a request. Without proper safeguards, the client won't know if the operation succeeded or failed, and a naive retry can execute the same business logic twice—charging a customer twice, creating duplicate invoices, or double-booking a reservation. Idempotency keys: a Design Patterns guide for developers solves this exact problem by ensuring that repeated requests with the same key always produce the same result, making your system resilient to network hiccups and retry storms.

In this post, we'll explore how idempotency keys work in practice, why they're essential for payment systems and critical mutations, and how to implement them across your stack.

Client sends a unique key per operation

The pattern begins with the client. Before initiating any operation that changes state—a payment, a transfer, a resource creation—the client generates a unique identifier and includes it in the request. This key must be stable for the same logical operation: if the network fails and the client retries, it sends the exact same key.

In practice, a client typically generates a UUID or uses a hash of the operation's parameters:

const crypto = require('crypto');
const axios = require('axios');

// Option 1: Generate a UUID (most common)
const idempotencyKey = crypto.randomUUID();

// Option 2: Deterministic hash (useful for deduplication)
function generateIdempotencyKey(userId, amount, recipientId) {
  const data = `${userId}-${amount}-${recipientId}-${Date.now()}`;
  return crypto.createHash('sha256').update(data).digest('hex');
}

const payload = {
  idempotencyKey: crypto.randomUUID(),
  amount: 25.00,
  currency: 'USD',
  recipient: 'acct_123456'
};

// Send to server with the key in the header
axios.post('/api/transfers', payload, {
  headers: {
    'Idempotency-Key': payload.idempotencyKey
  }
}).catch(error => {
  // On network error, retry with the SAME key
  console.log('Request failed, will retry with same key:', payload.idempotencyKey);
});

The key should be sent as an HTTP header (the standard approach) or in the request body. Most payment processors—Stripe, PayPal, Square—accept the Idempotency-Key header. Using a header keeps the key separate from your business payload and makes it easier to log and track across middleware layers.

Server stores the result for that key

When the server receives a request with an idempotency key, it doesn't immediately execute the business logic. Instead, it first checks whether it has already processed a request with that key. If the key exists in the idempotency store, it returns the cached result immediately without repeating the operation.

Here's a typical flow in a payment service:

import { Request, Response } from 'express';
import Redis from 'ioredis';

const redis = new Redis();

interface IdempotencyRecord {
  status: 'pending' | 'success' | 'error';
  result?: any;
  error?: string;
  createdAt: number;
}

async function handleTransfer(req: Request, res: Response) {
  const idempotencyKey = req.headers['idempotency-key'] as string;
  
  if (!idempotencyKey) {
    return res.status(400).json({ error: 'Idempotency-Key header required' });
  }

  // Check if we've already processed this key
  const cached = await redis.get(`idempotency:${idempotencyKey}`);
  
  if (cached) {
    const record: IdempotencyRecord = JSON.parse(cached);
    
    // Still processing: tell client to wait or retry
    if (record.status === 'pending') {
      return res.status(409).json({ error: 'Request in progress' });
    }
    
    // We've already succeeded: return the result
    if (record.status === 'success') {
      return res.status(200).json(record.result);
    }
    
    // We've already failed: return the same error
    if (record.status === 'error') {
      return res.status(400).json({ error: record.error });
    }
  }

  // Mark as pending to handle concurrent retries
  await redis.setex(
    `idempotency:${idempotencyKey}`,
    3600, // 1 hour TTL
    JSON.stringify({ status: 'pending', createdAt: Date.now() })
  );

  try {
    const { amount, recipientId } = req.body;
    
    // Execute the actual business logic
    const transferResult = await processTransfer(amount, recipientId);
    
    // Cache the successful result
    const record: IdempotencyRecord = {
      status: 'success',
      result: transferResult,
      createdAt: Date.now()
    };
    
    await redis.setex(
      `idempotency:${idempotencyKey}`,
      3600,
      JSON.stringify(record)
    );
    
    return res.status(200).json(transferResult);
    
  } catch (error: any) {
    // Cache the error
    const record: IdempotencyRecord = {
      status: 'error',
      error: error.message,
      createdAt: Date.now()
    };
    
    await redis.setex(
      `idempotency:${idempotencyKey}`,
      3600,
      JSON.stringify(record)
    );
    
    return res.status(400).json({ error: error.message });
  }
}

Using Redis here is popular because it's fast, supports TTLs (so you don't bloat storage), and handles concurrent requests well. You could also use PostgreSQL with a dedicated idempotency_logs table for stronger durability, or a combination of both (hot cache + persistent backup).

Retries return the original result

The beauty of this pattern is transparent to the client. If a network timeout occurs and the retry library sends the same request again with the same idempotency key, the server instantly recognizes it and returns the original result. No double charge, no duplicate resource, no wasted compute.

Consider a real scenario:

1. Client sends transfer request with key uuid-12345 and amount $100.
2. Server processes the transfer and credits the recipient.
3. Server is about to respond but crashes before the HTTP response is sent.
4. Client times out and retries with the same key.
5. Server recovers, checks Redis, finds key uuid-12345, and immediately returns the original result ($100 transferred).
6. Client receives the same response as if no crash had happened.

This pattern also handles aggressive retry logic gracefully. If a retry daemon keeps hammering your endpoint, the idempotency check short-circuits every duplicate request, protecting your database and payment processor from unnecessary load.

Essential for payments

Payment systems are the canonical use case. A single duplicate charge can trigger chargebacks, regulatory scrutiny, and customer support nightmares. Every major payment processor requires or strongly recommends idempotency keys for this reason.

In a real e-commerce flow:

// Client-side: checkout flow
async function processCheckout(cart) {
  const checkoutId = crypto.randomUUID();
  
  try {
    const response = await fetch('/api/checkout', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Idempotency-Key': checkoutId
      },
      body: JSON.stringify({
        items: cart.items,
        total: cart.total,
        customerId: getCurrentUserId()
      })
    });
    
    if (!response.ok) {
      throw new Error('Checkout failed');
    }
    
    const order = await response.json();
    return order;
    
  } catch (error) {
    // Retry with the SAME checkoutId
    console.error('Checkout error:', error);
    // A built-in retry mechanism will use the same headers
    return processCheckout(cart);
  }
}

Beyond payments, idempotency keys protect any critical mutation: account transfers, subscription sign-ups, inventory deductions, or booking confirmations. If your system can't tolerate duplicates, idempotency keys should be part of your design.

Common mistakes

1. Accepting a client-provided key without validation: Always ensure the idempotency key is properly formatted and reasonably sized (e.g., 36 characters for a UUID). Malicious or accidentally malformed keys can pollute your cache.

2. Storing idempotency results in memory without TTL: Your cache will grow indefinitely. Use Redis TTLs or database expiration (e.g., SETEX in Redis, ON DELETE CASCADE in PostgreSQL) to automatically clean up old records. A 24-hour window is typical; adjust based on your retry policy and compliance requirements.

3. Not handling the "pending" state: If two requests with the same key arrive concurrently, the first marks the key as "pending." The second should wait or return a 409 Conflict, not re-execute the logic. Without this guard, you risk duplicate execution under high concurrency.

4. Conflating idempotency keys with request IDs: Request IDs are for logging and tracing. Idempotency keys are for deduplication. Use both, but keep them separate. A request ID helps you find logs; an idempotency key ensures you don't charge twice.

5. Forgetting to cache errors: It's not enough to cache successful results. If a request fails, cache that failure too. When a retry comes in with the same key, return the same error immediately. This prevents confusion and repeated invalid API calls to downstream services.

Key takeaways

• Client generates a unique, stable key for each business operation and includes it in every request (typically as the Idempotency-Key header).
• Server checks the idempotency store before executing business logic. If the key exists, return the cached result immediately.
• Cache both success and error outcomes with a reasonable TTL (24 hours is common) to avoid indefinite storage growth.
• Handle the "pending" state to guard against concurrent duplicate execution when two identical requests arrive at the same time.
• Idempotency keys are non-negotiable for payment systems and any operation that cannot be safely repeated; they're equally valuable for any critical mutation in your system.
• Use Redis for speed or PostgreSQL for durability, or combine both. The right choice depends on your consistency and retention requirements.