DynamoDB with Terraform: real-world lessons and examples

DynamoDB with Terraform: real-world lessons and examples

We've all felt the pull: your application grows, relational databases start showing their seams, and suddenly you're looking at NoSQL. DynamoDB offers the scale and flexibility your team needs, but getting it right in production demands more than just spinning up a table. This guide covers the practical decisions we've made when managing DynamoDB with Terraform—from billing models to access patterns—drawing on real production scenarios across e-commerce, CRM, and ride-hailing platforms.

Starting with on-demand billing

One of the most expensive mistakes we see is provisioning DynamoDB capacity based on guesswork. Teams provision "just enough" during off-peak hours, then face throttling when traffic spikes at midnight. Conversely, they over-provision to be safe, then waste money when usage drops to nothing.

On-demand billing inverts this problem: you pay per request, no upfront capacity declaration. For tables that experience unpredictable or bursty traffic, this is usually the right starting point. You trade a predictable cost curve for automatic elasticity. In our experience, it makes sense for:

• New tables where access patterns aren't yet understood
• Notification and audit logs that spike during campaigns
• Backend services with variable load across regions
• Development and staging environments

Here's how we typically set it up:

resource "aws_dynamodb_table" "events_log" {
  name           = "events-log-${var.environment}"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "event_id"
  range_key      = "created_at"

  attribute {
    name = "event_id"
    type = "S"
  }

  attribute {
    name = "created_at"
    type = "N"
  }

  ttl {
    attribute_name = "expires_at"
    enabled        = true
  }

  tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
  }
}

The trade-off: on-demand costs roughly 1.5–2x per request compared to provisioned capacity at scale. Once you hit consistent throughput (we typically see this around 10k+ requests per second), provisioned capacity becomes cheaper. Monitor CloudWatch metrics for a few weeks before switching billing modes.

Point-in-time recovery for important tables

We learned this the hard way: a data migration bug corrupted a user profile table at 3 AM. Without backups, we were looking at manual restoration from logs. Now we treat PITR (point-in-time recovery) as mandatory for any table holding customer state.

PITR lets you restore a DynamoDB table to any point in the last 35 days. It's continuous and automatic—no snapshots to manage. The cost is roughly 20% of your table's on-demand write capacity, which is modest insurance for production data.

In Terraform, enabling it takes one line:

resource "aws_dynamodb_table" "users" {
  name           = "users-${var.environment}"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "user_id"

  attribute {
    name = "user_id"
    type = "S"
  }

  point_in_time_recovery {
    enabled = true
  }

  tags = {
    Environment = var.environment
    CriticalData = "true"
  }
}

But there's a nuance: you need a clear recovery runbook. If your table is corrupted at 3 AM, your team needs to know:

1. How to identify the corruption timestamp
2. Who can authorize a restore
3. How to validate the restored table before cutover
4. How to handle application state during the restore window

We typically reserve PITR for tables in the "critical data" category. Temporary tables—session caches, request queues—don't need it. Keep your infrastructure costs aligned with actual risk.

Global secondary indexes for access patterns

DynamoDB's killer feature is that you query by partition key (and optionally sort key) with blazing speed. But real applications need multiple access patterns. That's where global secondary indexes (GSIs) come in.

Imagine a ride-hailing platform. Your primary access pattern is get ride by ride_id. But you also need to query rides by driver_id or by status. Without GSIs, you'd scan the entire table—expensive and slow. With GSIs, you create alternate key structures:

resource "aws_dynamodb_table" "rides" {
  name           = "rides-${var.environment}"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "ride_id"

  attribute {
    name = "ride_id"
    type = "S"
  }

  attribute {
    name = "driver_id"
    type = "S"
  }

  attribute {
    name = "status"
    type = "S"
  }

  attribute {
    name = "created_at"
    type = "N"
  }

  # Primary table: query by ride_id
  global_secondary_index {
    name            = "driver_id-created_at-gsi"
    hash_key        = "driver_id"
    range_key       = "created_at"
    projection_type = "ALL"
  }

  # Sparse index: only active rides
  global_secondary_index {
    name            = "status-created_at-gsi"
    hash_key        = "status"
    range_key       = "created_at"
    projection_type = "INCLUDE"
    non_key_attributes = ["driver_id", "passenger_id", "fare"]
  }

  tags = {
    Environment = var.environment
  }
}

A few practical notes on GSIs:

Projection type matters: ALL projects every attribute (slower writes, easier queries). INCLUDE projects only specified attributes (faster writes, requires fetch on miss). KEYS_ONLY projects only the keys (fastest writes, most fetches).

Sparse indexes save money: If your index only covers 10% of rows (e.g., only active rides), use a sparse index. DynamoDB only charges for data you actually store. Add a partition key condition in your code to prevent scanning the whole thing.

GSI throughput is separate: In provisioned mode, GSI throughput comes from a separate capacity pool. In on-demand mode, you don't worry about it. But design your GSIs up front; adding one to a live table is transparent to clients but takes time.

TTL attributes for expiring data

Session tokens, temporary OTPs, rate-limit counters—these need to disappear automatically. DynamoDB's TTL feature does exactly that: it deletes items when a specified timestamp passes.

The catch: TTL is eventual consistency. Items may linger 48 hours past expiry. For security-sensitive data, validate manually. For convenience data (session cache), TTL is perfect.

Here's a real example from a travel booking platform:

resource "aws_dynamodb_table" "booking_sessions" {
  name           = "booking-sessions-${var.environment}"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "session_id"

  attribute {
    name = "session_id"
    type = "S"
  }

  ttl {
    attribute_name = "expires_at"
    enabled        = true
  }

  tags = {
    Environment = var.environment
  }
}

When you write a session, include an expires_at timestamp (Unix epoch seconds):

{
  "session_id": "sess_12345",
  "user_id": "user_789",
  "booking_state": {...},
  "expires_at": 1700000000
}

DynamoDB scans for expired items in the background. Once they pass the TTL threshold, they're eligible for deletion. There's no charge for TTL deletions—they're just automatic cleanup.

Pro tip: Always include a TTL on session and cache tables. It prevents your database from becoming a graveyard of stale data. For audit tables that need indefinite retention, skip TTL but enable PITR instead.

Common mistakes

Not monitoring DynamoDB Contributor Insights: This feature (free) shows you which partitions are hot and which queries are slow. Enable it and check the CloudWatch dashboard. We've caught N+1 query patterns and hot partitions before they became incidents.

Forgetting about item size limits: DynamoDB items max out at 400 KB. Storing serialized JSON blobs without compression can sneak up on you. Use compression or normalize your schema.

Creating too many GSIs: Each GSI slows your writes (it must be updated synchronously). Aim for 3–4 indexes per table. If you need more access patterns, consider denormalizing differently or using a second table.

Ignoring local secondary indexes: LSIs share throughput with the base table and have a 10 GB size limit per partition key. They're useful for exact-time-window queries but rarely the right choice today. GSIs are almost always better.

Not testing restore procedures: PITR looks great on paper. But if you've never actually restored a table and validated the data, you'll be learning in production. Run a restore drill quarterly.

Key takeaways

• Start with on-demand billing for new tables; switch to provisioned once you understand your access pattern and throughput is consistent.
• Enable PITR on any table holding customer state or business-critical data. It's cheap insurance against application bugs.
• Design your GSIs around your actual query patterns before launch. Add indexes thoughtfully—each one slows writes.
• Use TTL for transient data: sessions, caches, temporary tokens. Let DynamoDB clean up automatically.
• Monitor Contributor Insights and CloudWatch metrics weekly. Hot partitions and slow queries emerge fast as traffic grows.
• Test your restore procedures in staging. A recovery runbook is only useful if your team has practiced it.