Idempotency: the API property nobody teaches in practice

Idempotency: the API property nobody teaches

Networks fail. Users panic and hit refresh. Webhooks retry after timeout. In all of these scenarios, your API receives the same request twice—and if you haven't built for idempotency, you've just charged the customer twice, created duplicate orders, or sent the same notification three times. Idempotency: the API property nobody teaches because most tutorials skip it, yet it's non-negotiable in production systems that handle money, inventory, or critical state changes.

The same request twice should have the effect of once

Idempotency is simple in principle: a request should produce the same result regardless of how many times you send it. Submit a payment twice? One charge. Place an order twice? One order created. This is different from read operations—fetching a user is naturally idempotent. The problem lives in write operations: POST /payments, POST /orders, POST /refunds.

Without idempotency, you're relying on networks to never fail. In reality:

• The client sends a request and receives no response (timeout).
• The server processes the request successfully but crashes before returning the response.
• The response is lost in transit to the client.
• The user double-clicks the submit button.
• A webhook delivery retry mechanism sends the same event again.

In each case, the client resends the request. If you process it again, you've now created duplicate side effects. I've seen this cause:

• Double charges on payment processing (nightmare for support).
• Duplicate orders appearing in analytics (data corruption).
• Double-deducted inventory (overselling).
• Duplicate webhook events triggering downstream actions.

The solution is to make your API idempotent: design it so that processing the same request multiple times is safe.

Clients send an idempotency key per operation

The practical mechanism is the idempotency key: a unique identifier that the client generates and includes with every request that should be idempotent. The server uses this key to recognize retries and avoid reprocessing.

The client generates a UUID and sends it in the request:

// Client-side: POST /payments
const idempotencyKey = crypto.randomUUID();

const response = await fetch('https://api.example.com/payments', {
  method: 'POST',
  headers: {
    'Idempotency-Key': idempotencyKey,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    amount: 2500,
    currency: 'USD',
    customerId: 'cust_123',
  }),
});

// If the network fails, retry with the same Idempotency-Key
const retryResponse = await fetch('https://api.example.com/payments', {
  method: 'POST',
  headers: {
    'Idempotency-Key': idempotencyKey, // Same key
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    amount: 2500,
    currency: 'USD',
    customerId: 'cust_123',
  }),
});

The key should be unique per logical operation. For a payment, use one key per checkout submission. For an order, one key per order creation. If the user clicks submit, submit again, and submit again, all three requests should carry the same idempotency key.

The standard header name is Idempotency-Key, though some systems use X-Idempotency-Key or encode it in the request body. Stripe, AWS, and most payment processors use Idempotency-Key in the HTTP header. Follow that convention unless you have a reason not to.

On the server side, you extract and validate the key:

// NestJS example
import { Controller, Post, Headers, Body } from '@nestjs/common';

@Controller('payments')
export class PaymentController {
  constructor(private paymentService: PaymentService) {}

  @Post()
  async createPayment(
    @Headers('idempotency-key') idempotencyKey: string,
    @Body() createPaymentDto: CreatePaymentDto,
  ) {
    if (!idempotencyKey) {
      throw new BadRequestException('Idempotency-Key header is required');
    }
    return this.paymentService.processPayment(idempotencyKey, createPaymentDto);
  }
}

The idempotency key must be required for sensitive operations. If the client doesn't send one, reject the request. This enforces the contract: idempotency is not optional for payments.

Store the key and response; replay on retries

When the server receives a request with an idempotency key, it checks if that key has been processed before. If yes, return the cached response. If no, process the request, store the result, and return it.

Create a table to track idempotency:

Now implement the logic in your payment service:

// PaymentService
import { Injectable } from '@nestjs/common';
import { Repository } from 'typeorm';
import { InjectRepository } from '@nestjs/typeorm';

@Injectable()
export class PaymentService {
  constructor(
    @InjectRepository(IdempotencyRecord)
    private idempotencyRepo: Repository,
  ) {}

  async processPayment(
    idempotencyKey: string,
    createPaymentDto: CreatePaymentDto,
  ) {
    // Step 1: Check if this key has been processed
    const existing = await this.idempotencyRepo.findOne({
      where: { idempotency_key: idempotencyKey, operation: 'payment' },
    });

    if (existing) {
      // Return cached response
      return {
        statusCode: existing.status === 'success' ? 200 : 400,
        body: existing.response_body,
        isRetry: true,
      };
    }

    // Step 2: Record that we're processing this key (mark as pending)
    const record = this.idempotencyRepo.create({
      idempotency_key: idempotencyKey,
      operation: 'payment',
      status: 'pending',
      response_body: null,
      expires_at: new Date(Date.now() + 24 * 60 * 60 * 1000), // 24 hours
    });
    await this.idempotencyRepo.save(record);

    try {
      // Step 3: Process the payment
      const payment = await this.chargeCard(createPaymentDto);

      // Step 4: Store the response
      record.status = 'success';
      record.response_body = {
        paymentId: payment.id,
        amount: payment.amount,
        status: payment.status,
      };
      await this.idempotencyRepo.save(record);

      return {
        statusCode: 201,
        body: record.response_body,
        isRetry: false,
      };
    } catch (error) {
      // Step 5: Store the error
      record.status = 'failed';
      record.response_body = { error: error.message };
      await this.idempotencyRepo.save(record);
      throw error;
    }
  }

  private async chargeCard(dto: CreatePaymentDto) {
    // Call your payment processor (Stripe, etc.)
    // This only runs once, on the first request.
  }
}

Key points in this flow:

• Check first: Always look up the idempotency key before processing.
• Mark pending: Save a "pending" record immediately to detect concurrent retries (two requests arrive before the first finishes).
• Process once: Only call the payment processor, database write, or external API if the key is new.
• Cache the response: Store the full response body, including success or error.
• Return cached response on retry: Don't re-process; return what you stored.
• Handle concurrent retries: If two requests arrive with the same key simultaneously, the second should wait for the first to complete, then return the cached result. Use a database lock or distributed lock (Redis) for this.

With Redis, you can add a simple check-and-lock pattern:

// Using Redis for locking during concurrent retries
async processPaymentWithLock(
  idempotencyKey: string,
  createPaymentDto: CreatePaymentDto,
) {
  const lockKey = `payment_lock:${idempotencyKey}`;
  const lockValue = crypto.randomUUID();

  // Try to acquire lock
  const locked = await this.redis.set(
    lockKey,
    lockValue,
    'EX',
    30, // 30 second timeout
    'NX', // Only if not exists
  );

  if (!locked) {
    // Another request is processing this key, wait and retry
    await new Promise(resolve => setTimeout(resolve, 500));
    return this.processPaymentWithLock(idempotencyKey, createPaymentDto);
  }

  try {
    return await this.processPayment(idempotencyKey, createPaymentDto);
  } finally {
    await this.redis.del(lockKey);
  }
}

Non-negotiable for payments, orders and webhooks

Idempotency is mandatory for three categories of operations:

Payments and refunds: Charging a customer twice is your company's liability. If a user clicks "Pay" twice and both go through, you're issuing a refund and apologizing. With idempotency, the second request returns the same transaction ID, and no charge occurs. Stripe, PayPal, and every serious payment processor requires an idempotency key for card charges.

Orders and transactions: When a user places an order, creating it twice means duplicate inventory deduction, duplicate shipments, and data integrity chaos. One order per submission, enforced by idempotency.

Webhooks: Your system sends webhooks to third-party integrations (Slack, email service, CRM). If the webhook delivery fails and you retry, the receiver must handle receiving the same event twice. Stripe solves this by including an event ID; receivers should store event IDs and skip duplicates. You should do the same for your outbound webhooks: include an event ID header and expect receivers to deduplicate.

For less critical operations like analytics or logging, idempotency is nice-to-have but not essential. For anything touching money, inventory, or state mutation, it's non-negotiable.

Common mistakes

Mistake 1: Making idempotency optional. If the client doesn't send an idempotency key, you allow the request anyway. This creates a two-tier system: safe requests (with key) and unsafe requests (without key). Require the key for sensitive operations. Stripe rejects payment requests without an idempotency key; you should too.

Mistake 2: Storing only the response body, not the status code. If a request fails (e.g., card declined), you must cache that failure too. On retry, return the same error, not retry the charge. Your idempotency record should store both success and failure.

Mistake 3: Not handling concurrent retries. If two requests with the same key arrive within milliseconds, both might start processing before either stores the result. Use a lock (database or Redis) to ensure only one processes; the other waits and reads the result.

Mistake 4: Never cleaning up old records. After 24 hours, you can safely delete the idempotency record. A client retrying after 24 hours is rare; you're wasting storage. Set an expires_at timestamp and delete old records periodically.

Mistake 5: Applying idempotency to GET requests. GET is already idempotent by definition. Only protect POST, PATCH, PUT, and DELETE operations that mutate state.

Key takeaways

• Idempotency ensures the same request has the effect of being processed once, protecting against network failures, retries, and user error.
• Require clients to send a unique idempotency key in the request header for all write operations on sensitive resources.
• Store each key with its response (success or failure) in a database or cache. On retry, return the cached response without reprocessing.
• Handle concurrent retries with a lock (Redis or database) so only one request processes; others wait and read the cached result.
• Idempotency is non-negotiable for payments, orders, refunds, and any operation that mutates critical state or inventory.
• Clean up old idempotency records (older than 24 hours) to save storage; clients rarely retry after that window.
• Test idempotency by simulating network failures and duplicate submissions; verify that the second request returns the same result without side effects.