When a ride-hailing service books a vehicle, charges a payment, and confirms the booking across three microservices, what happens if the payment fails halfway through? Traditional two-phase commit locks resources and doesn't scale in distributed systems. The saga pattern: a Design Patterns guide for developers offers a practical alternative—a way to manage long-running transactions by breaking them into steps, each with a compensating action ready to undo its work if needed.
Understanding the saga pattern
A saga is a sequence of local transactions across multiple services. Unlike traditional ACID transactions that lock data, a saga commits each step independently and relies on compensating transactions—explicit undo operations—to maintain consistency. This approach works well in microservices architectures where services own their data and network calls are expected to fail.
The key insight is that we trade immediate consistency for availability and resilience. Instead of asking the database "are all these operations guaranteed to succeed together?" we ask "if any step fails, do we have a clear path to undo what we've done?" This shift enables systems to remain responsive even when individual services are slow or temporarily unavailable.
The booking saga: a concrete example
Let's walk through a real booking flow across four services: vehicle inventory, payment, booking confirmation, and notifications.
- Reserve vehicle: The inventory service locks a vehicle and reserves it for 30 minutes.
- Charge payment: The payment service charges the customer's card.
- Confirm booking: The booking service creates a booking record and assigns the vehicle.
- Send confirmation: The notification service sends an email.
If the payment fails at step 2, we need to release the vehicle reservation. If the confirmation fails at step 3, we need to refund the payment and release the vehicle. Each step must know how to undo itself.
Here's a simple state machine tracking the saga's progress:
interface SagaStep {
name: string;
execute: (context: any) => Promise<any>;
compensate: (context: any) => Promise<void>;
}
interface SagaState {
id: string;
status: 'PENDING' | 'IN_PROGRESS' | 'COMPLETED' | 'FAILED' | 'COMPENSATING';
completedSteps: string[];
failedStep?: string;
createdAt: Date;
updatedAt: Date;
}
class BookingSaga {
private steps: SagaStep[] = [
{
name: 'reserve_vehicle',
execute: async (ctx) => {
const reservation = await inventoryService.reserve(ctx.vehicleId);
ctx.reservationId = reservation.id;
},
compensate: async (ctx) => {
await inventoryService.release(ctx.reservationId);
}
},
{
name: 'charge_payment',
execute: async (ctx) => {
const charge = await paymentService.charge(ctx.customerId, ctx.amount);
ctx.chargeId = charge.id;
},
compensate: async (ctx) => {
await paymentService.refund(ctx.chargeId);
}
},
{
name: 'confirm_booking',
execute: async (ctx) => {
const booking = await bookingService.create({
customerId: ctx.customerId,
vehicleId: ctx.vehicleId,
reservationId: ctx.reservationId
});
ctx.bookingId = booking.id;
},
compensate: async (ctx) => {
await bookingService.cancel(ctx.bookingId);
}
},
{
name: 'notify_customer',
execute: async (ctx) => {
await notificationService.sendConfirmation(ctx.customerId, ctx.bookingId);
},
compensate: async (ctx) => {
await notificationService.sendCancellation(ctx.customerId, ctx.bookingId);
}
}
];
async execute(context: any): Promise<SagaState> {
const sagaId = generateId();
const state: SagaState = {
id: sagaId,
status: 'PENDING',
completedSteps: [],
createdAt: new Date(),
updatedAt: new Date()
};
// Persist initial state
await sagaStore.save(state);
try {
state.status = 'IN_PROGRESS';
await sagaStore.save(state);
for (const step of this.steps) {
try {
await step.execute(context);
state.completedSteps.push(step.name);
await sagaStore.save(state);
} catch (error) {
state.failedStep = step.name;
state.status = 'COMPENSATING';
await sagaStore.save(state);
// Run compensations in reverse order
for (let i = state.completedSteps.length - 1; i >= 0; i--) {
const completedStepName = state.completedSteps[i];
const stepToCompensate = this.steps.find(s => s.name === completedStepName);
try {
await stepToCompensate.compensate(context);
} catch (compError) {
// Log for manual intervention
console.error(`Failed to compensate ${completedStepName}`, compError);
}
}
state.status = 'FAILED';
await sagaStore.save(state);
throw error;
}
}
state.status = 'COMPLETED';
await sagaStore.save(state);
return state;
} catch (error) {
throw new Error(`Saga ${sagaId} failed: ${error.message}`);
}
}
}
Compensating actions: the undo mechanism
The heart of a saga is its compensating transaction. For every forward action, we must define a precise undo. This isn't a generic rollback—it's a deliberate, explicit operation that knows exactly what state to clean up.
Consider three patterns:
- Credit/debit: If we debit an account, the compensation credits it back.
- Create/delete: If we create a reservation, the compensation deletes it (or marks it expired).
- Update/revert: If we update a status, the compensation reverts to the previous status.
A compensation is not always a true inverse. In payment refunds, we don't reverse the original charge—we issue a separate refund transaction. In vehicle reservations, we mark the reservation as cancelled rather than deleting the record. Compensation must be idempotent: running it twice should have the same effect as running it once. This is crucial because network failures may cause compensations to be retried.
Here's an example of an idempotent refund:
async refund(chargeId: string): Promise<void> {
const charge = await db.charges.findById(chargeId);
// If already refunded, do nothing
if (charge.status === 'REFUNDED') {
return;
}
// If refund was started but not completed, resume it
if (charge.status === 'REFUNDING') {
const refundRecord = await db.refunds.findByChargeId(chargeId);
if (refundRecord?.status === 'COMPLETED') {
await db.charges.update(chargeId, { status: 'REFUNDED' });
return;
}
}
// Issue the refund
const refundRecord = await paymentGateway.refund(charge.externalId);
await db.refunds.insert({ chargeId, externalRefundId: refundRecord.id });
await db.charges.update(chargeId, { status: 'REFUNDING' });
// Confirm completion
await db.charges.update(chargeId, { status: 'REFUNDED' });
}
Choreography versus orchestration
There are two ways to coordinate saga steps: choreography and orchestration. Each has trade-offs.
Choreography: event-driven coordination
Each service listens for events and emits new events when its step completes. There's no central coordinator—services react to each other through an event bus (Kafka, RabbitMQ, or SNS).
// Inventory service: emits when vehicle is reserved
inventoryService.on('reserve', async (vehicleId) => {
const reservation = await reserve(vehicleId);
eventBus.emit('vehicle_reserved', { vehicleId, reservationId: reservation.id });
});
// Payment service: listens for vehicle_reserved, then emits payment event
eventBus.on('vehicle_reserved', async (event) => {
try {
const charge = await paymentService.charge(event.customerId, event.amount);
eventBus.emit('payment_charged', { ...event, chargeId: charge.id });
} catch (error) {
eventBus.emit('payment_failed', { ...event, reason: error.message });
}
});
// Inventory service: listens for payment failure and compensates
eventBus.on('payment_failed', async (event) => {
await inventoryService.release(event.reservationId);
});
Pros: Loose coupling, each service is independent, easy to add new participants.
Cons: Complex to debug (events flow through many services), hard to track saga progress, difficult to enforce saga semantics (what if two services emit conflicting events?).
Orchestration: centralized coordination
A dedicated orchestrator service calls each step in sequence and handles compensation if a step fails. The orchestrator holds the saga logic and state.
// Orchestrator: controls the entire saga
class BookingOrchestrator {
async startSaga(booking: BookingRequest): Promise<void> {
const saga = await this.createSaga(booking);
try {
// Step 1: Reserve
saga.reservationId = await inventoryService.reserve(booking.vehicleId);
// Step 2: Charge
saga.chargeId = await paymentService.charge(booking.customerId, booking.amount);
// Step 3: Confirm
saga.bookingId = await bookingService.confirm(saga);
// Step 4: Notify
await notificationService.send(saga);
saga.status = 'COMPLETED';
} catch (error) {
await this.compensate(saga);
saga.status = 'FAILED';
}
await this.persistSaga(saga);
}
private async compensate(saga: Saga): Promise<void> {
if (saga.bookingId) await bookingService.cancel(saga.bookingId);
if (saga.chargeId) await paymentService.refund(saga.chargeId);
if (saga.reservationId) await inventoryService.release(saga.reservationId);
}
}
Pros: Easy to understand and debug, clear saga flow, simple to persist state and retry.
Cons: Central orchestrator is a single point of failure, tighter coupling to service APIs, orchestrator becomes a bottleneck as complexity grows.
In practice, orchestration works well for booking and checkout flows with a clear linear sequence. Choreography scales better when there are many asynchronous participants and events flow in multiple directions.
Tracking state: persisting saga progress
A saga must survive service restarts and network partitions. We track every state transition in durable storage. When a service crashes mid-saga, we resume from the last completed step.
Store the saga state in a database table:
| Column | Type | Purpose |
|---|---|---|
| id | UUID | Unique saga identifier |
| status | ENUM | PENDING, IN_PROGRESS, COMPLETED, COMPENSATING, FAILED |
| context | JSONB | Booking details, customer ID, vehicle ID, reservation ID, charge ID |
| completed_steps | TEXT[] | Array of step names that succeeded |
| failed_step | TEXT | Step name that failed (if any) |
| created_at | TIMESTAMP | When the saga started |
| updated_at | TIMESTAMP | Last state change |
With this persisted state, we can implement retries and recovery:
async resumeSaga(sagaId: string): Promise<void> {
const saga = await sagaStore.findById(sagaId);
if (saga.status === 'COMPLETED' || saga.status === 'FAILED') {
return; // Nothing to do
}
if (saga.status === 'COMPENSATING') {
// Resume compensation
await this.compensate(saga);
} else if (saga.status === 'IN_PROGRESS') {
// Find the next unstarted step and resume
const nextStepIndex = saga.completedSteps.length;
await this.executeFromStep(saga, nextStepIndex);
}
}
async executeFromStep(saga: Saga, stepIndex: number): Promise<void> {
const steps = this.getSteps();
for (let i = stepIndex; i < steps.length; i++) {
const step = steps[i];
try {
await step.execute(saga.context);
saga.completedSteps.push(step.name);
await sagaStore.update(sagaId, saga);
} catch (error) {
saga.failedStep = step.name;
saga.status = 'COMPENSATING';
await sagaStore.update(sagaId, saga);
throw error;
}
}
saga.status = 'COMPLETED';
await sagaStore.update(sagaId, saga);
}
This design also enables timeout handling. If a saga hasn't progressed in 30 minutes, a background job can check its status and retry or escalate it for manual review.
Common mistakes
- Forgetting idempotency: If a step runs twice (due to a retry), the second run must have the same effect as the first. Use idempotency keys and check state before executing.
- Losing saga state: If you don't persist saga progress, a crash during compensation will leave the system in an inconsistent state. Always save state before each step and before starting compensation.
- Assuming compensation always works: A compensation may fail (payment refund is declined, database is down). Log these failures and alert on-call engineers—manual intervention is sometimes necessary.
- Mixing patterns carelessly: Using orchestration for some steps and choreography for others creates confusion. Pick one pattern per saga.
- Not defining clear timeout boundaries: If a step takes too long, when do we compensate? Define and document timeout policies for each step.
- Treating sagas as transactions: Sagas don't guarantee isolation. Other services may observe partial results (e.g., a charged payment before the booking is confirmed). Design your UI and downstream logic to handle this.
Key takeaways
- A saga breaks a distributed transaction into steps, each with a compensating action, enabling consistency without two-phase commit.
- Every forward action must have a precise, idempotent undo operation defined before the saga starts.
- Orchestration (central coordinator) is simpler to understand and manage for booking flows; choreography (event-driven) scales when many services participate asynchronously.
- Persist saga state to a durable store after each step so you can resume after failures and recover from restarts.
- Compensation may fail and require human intervention—log thoroughly and design alerts for partial failures.
- Sagas trade immediate consistency for availability; your API and UI must tolerate seeing partial results temporarily.
Comments
Loading comments…