Lambda patterns I use: a practical guide for developers

After running dozens of workloads across production systems, we've settled on a handful of Lambda patterns that solve real problems without the complexity of managing servers. This guide covers the Lambda patterns I use: a practical guide for developers who want to move faster without betting the farm on serverless.

API backends

The most common use case. API Gateway routes HTTP requests to Lambda functions, which is perfect when your traffic is spiky or when you're building a new endpoint that might never be called.

A typical scenario: you're building a SaaS dashboard. During business hours, thousands of users hit the API. At night, traffic drops to near zero. With traditional servers, you're paying for capacity that sits idle. Lambda scales to zero when no one's using it.

The pattern is straightforward. API Gateway receives a request, invokes your Lambda, and returns the response. You don't manage connections or capacity.

// Handler receives event from API Gateway
exports.handler = async (event) => {
  const userId = event.pathParameters.id;
  const user = await db.getUser(userId);
  
  return {
    statusCode: 200,
    body: JSON.stringify(user),
    headers: { 'Content-Type': 'application/json' }
  };
};

One thing we learned: cold starts matter. The first invocation takes 100–500ms longer because the runtime needs to initialize. For user-facing APIs, this is noticeable. Solutions include provisioned concurrency (costs money but eliminates cold starts) or keeping the function warm with scheduled pings. We usually accept cold starts on internal APIs and use provisioned concurrency only when latency is critical.

API Gateway also lets you set up request validation, throttling, and CORS at the gateway level rather than inside your function. That's where it belongs.

Queue workers

The second pattern we rely on heavily. Instead of processing work synchronously, you put messages on an SQS queue and Lambda processes them asynchronously. This decouples the API from the work, which means your user gets a response immediately while the real work happens in the background.

Real scenario: a user uploads a CSV with 10,000 customer records. You validate the upload and return success within 100ms. Lambda polls SQS, picks up the message, and processes it in batches over the next few minutes. If something fails halfway through, SQS retries automatically.

The Lambda event source mapping configuration handles the polling for you. AWS watches the queue and invokes Lambda when messages arrive.

// Lambda triggered by SQS batch
exports.handler = async (event) => {
  const results = [];
  
  for (const record of event.Records) {
    try {
      const payload = JSON.parse(record.body);
      const result = await processCustomer(payload);
      results.push({ id: record.messageId, status: 'success' });
    } catch (err) {
      // Mark for retry without throwing
      results.push({ id: record.messageId, status: 'failed' });
    }
  }
  
  // SQS skips successfully processed messages
  // Failed ones go back to the queue
  return { batchItemFailures: results.filter(r => r.status === 'failed') };
};

Partial failure handling is crucial here. If you process 100 messages and 5 fail, you want those 5 to retry without reprocessing the 95 that succeeded. Return batchItemFailures with the message IDs that failed. SQS automatically returns those to the queue for retry.

We set SQS visibility timeout to be longer than your expected execution time. If Lambda takes 30 seconds on average, set visibility timeout to 90 seconds. If the function crashes, the message becomes visible again after timeout and gets retried.

Scheduled jobs

EventBridge Scheduler (the newer approach, though CloudWatch Events works too) lets you run Lambda on a schedule without managing cron servers. This replaces traditional cron jobs.

Examples: daily backups, hourly analytics aggregation, nightly cleanup of orphaned database records, weekly report generation. Any task that runs on a timer.

Configuration is simple. You create a schedule expression in EventBridge and point it at your Lambda. The execution happens automatically.

// Handler runs daily at 2 AM UTC
exports.handler = async (event) => {
  console.log('Daily job starting...');
  
  const yesterday = new Date();
  yesterday.setDate(yesterday.getDate() - 1);
  
  const logs = await analytics.getDailyLogs(yesterday);
  const summary = await summarize(logs);
  
  await db.saveReport({ date: yesterday, summary });
  
  return { statusCode: 200, processed: logs.length };
};

The big win: no separate server running all the time waiting for the scheduled moment. The Lambda function exists only when it runs. You're charged only for execution time.

We always add retry logic and dead-letter handling. If a scheduled job fails, EventBridge can retry it or send it to an SQS queue for manual inspection. This prevents silent failures.

File processing

S3 can trigger Lambda functions when files are uploaded. This is where Lambda shines for lightweight data transformation — image resizing, document parsing, log analysis, thumbnail generation.

Scenario: users upload photos to your S3 bucket. Lambda automatically resizes them into three formats (thumbnail, medium, large) and stores them alongside the original. No user waits for the resize; it happens after the upload completes.

S3 sends an event to Lambda with metadata about the uploaded file. Your function reads from S3, transforms the data, and writes back to S3 or a database.

const AWS = require('aws-sdk');
const sharp = require('sharp');

const s3 = new AWS.S3();

exports.handler = async (event) => {
  // S3 puts event in Records array
  for (const record of event.Records) {
    const bucket = record.s3.bucket.name;
    const key = decodeURIComponent(record.s3.object.key);
    
    // Get the original image
    const obj = await s3.getObject({ Bucket: bucket, Key: key }).promise();
    const buffer = obj.Body;
    
    // Generate resized versions
    const thumbnail = await sharp(buffer).resize(100, 100).toBuffer();
    const medium = await sharp(buffer).resize(400, 400).toBuffer();
    
    // Store resized images
    await s3.putObject({
      Bucket: bucket,
      Key: `thumbnails/${key}`,
      Body: thumbnail
    }).promise();
    
    await s3.putObject({
      Bucket: bucket,
      Key: `medium/${key}`,
      Body: medium
    }).promise();
  }
};

One consideration: Lambda timeout. For large files or heavy processing, you might exceed Lambda's 15-minute maximum execution time. We've hit this with PDF parsing of massive documents. The solution is breaking the work into smaller pieces or using a different service like Fargate for longer-running tasks.

Also be careful with S3 event recursion. If your Lambda writes to the same bucket that triggered it, you can create a loop. Use key prefixes in your event filter to avoid re-triggering on your own writes.

Common mistakes

• Ignoring cold starts in user-facing APIs. A 300ms cold start feels slow to users. Test production traffic patterns and use provisioned concurrency if needed.
• Not handling partial failures in queue workers. Throwing an exception on one message will cause the entire batch to retry, even though 99 others succeeded. Always return batchItemFailures.
• Forgetting Lambda environment limits. Temporary storage is 512MB. Deployment package is 250MB (50MB zipped). Large workloads need careful planning or a different service.
• Setting timeouts too low. A 3-second timeout sounds safe until your database is slow one day. Be conservative; you can always adjust down after production runs.
• Missing error handling and logging. Lambda runs somewhere you don't see. Add structured logging (JSON format for CloudWatch) and catch errors explicitly. Silent failures hide problems.

Key takeaways

• Lambda patterns I use revolve around four main scenarios: APIs with variable load, background work via queues, scheduled tasks, and file processing — each solves a specific operational problem.
• Cold starts are real but manageable; test production load and use provisioned concurrency when user experience matters.
• Queue workers must handle partial failures by returning batchItemFailures, not throwing exceptions.
• Scheduled jobs replace cron servers; you pay only for execution, not idle time.
• File processing triggers are powerful for transformation but watch timeout limits on heavy workloads.
• Always log structured data, set timeouts conservatively, and test error paths in production-like conditions.