EventBridge with Terraform: patterns, pitfalls and fixes

EventBridge sits at the heart of many event-driven architectures, yet teams often struggle to codify its configuration reliably. We've seen production incidents stemming from misconfigured rules, missing dead-letter queues, and targets that silently fail. In this guide, we'll walk through EventBridge with Terraform: patterns, pitfalls and fixes, covering rule matching, target routing, scheduled rules, and resilience patterns that keep events flowing.

Rules match event patterns

EventBridge rules are pattern matchers. You define a rule with an event pattern, and every event that passes through your event bus gets evaluated against that pattern. If it matches, the rule's targets are invoked. This is where most teams go wrong: they either write patterns that are too broad (matching everything) or too narrow (matching nothing).

Let's say you're building an e-commerce platform where order events flow through a central event bus. You want to route "order.created" events to one handler and "order.failed" events to another. In Terraform, you'd define rules like this:

resource "aws_cloudwatch_event_rule" "order_created" {
  name           = "order-created-rule"
  event_bus_name = aws_cloudwatch_event_bus.orders.name
  
  event_pattern = jsonencode({
    source      = ["order.service"]
    detail-type = ["Order Created"]
    detail = {
      status = ["pending"]
    }
  })
}

resource "aws_cloudwatch_event_target" "order_created_handler" {
  rule      = aws_cloudwatch_event_rule.order_created.name
  event_bus_name = aws_cloudwatch_event_bus.orders.name
  arn       = aws_lambda_function.process_order.arn
  role_arn  = aws_iam_role.eventbridge_invoke_lambda.arn
}

resource "aws_lambda_permission" "allow_eventbridge" {
  statement_id  = "AllowExecutionFromEventBridge"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.process_order.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.order_created.arn
}

Notice the jsonencode() wrapper around the event pattern. This is critical—it ensures your HCL map gets serialized into valid JSON that EventBridge understands. The pattern itself uses the source and detail-type fields to narrow down which events reach your target.

One subtle pitfall: if your pattern uses nested logic (like matching on specific order statuses), ensure your event producers actually emit those fields. If the "status" field is missing from incoming events, your rule silently won't match anything, and debugging becomes a hunt through CloudWatch Logs.

Targets: Lambda, SQS, Step Functions

Once a rule matches, EventBridge invokes your targets. The three we see most often in production are Lambda functions (for immediate processing), SQS queues (for async work or buffering), and Step Functions (for orchestrated workflows). Each has trade-offs.

Lambda targets work well for simple, fast operations—validating data, sending notifications, or kicking off database queries. But if your Lambda takes too long or fails, EventBridge retries for up to 24 hours by default, which can overwhelm your function. SQS gives you a buffer: events queue up, your workers consume at their own pace, and you get visibility into backlog.

Here's a practical setup with multiple targets:

resource "aws_cloudwatch_event_rule" "payment_processed" {
  name           = "payment-processed"
  event_bus_name = aws_cloudwatch_event_bus.payments.name
  
  event_pattern = jsonencode({
    source      = ["payment.gateway"]
    detail-type = ["Payment Processed"]
  })
}

# Target 1: Lambda for validation
resource "aws_cloudwatch_event_target" "validate_payment" {
  rule      = aws_cloudwatch_event_rule.payment_processed.name
  event_bus_name = aws_cloudwatch_event_bus.payments.name
  arn       = aws_lambda_function.validate_payment.arn
  role_arn  = aws_iam_role.eventbridge_invoke_lambda.arn
  
  dead_letter_config {
    arn = aws_sqs_queue.payment_dlq.arn
  }
}

# Target 2: SQS for async fulfillment
resource "aws_cloudwatch_event_target" "fulfill_order" {
  rule      = aws_cloudwatch_event_rule.payment_processed.name
  event_bus_name = aws_cloudwatch_event_bus.payments.name
  arn       = aws_sqs_queue.fulfillment.arn
  role_arn  = aws_iam_role.eventbridge_send_sqs.arn
}

# Target 3: Step Functions for complex workflow
resource "aws_cloudwatch_event_target" "audit_workflow" {
  rule      = aws_cloudwatch_event_rule.payment_processed.name
  event_bus_name = aws_cloudwatch_event_bus.payments.name
  arn       = aws_sfn_state_machine.audit_trail.arn
  role_arn  = aws_iam_role.eventbridge_invoke_step_functions.arn
}

Notice that one rule can have multiple targets. EventBridge invokes all matching targets in parallel (or nearly so)—there's no guaranteed order. This is a common surprise: teams expect sequential execution but get concurrent calls. Design your targets to be idempotent if you care about order or retries.

Also notice the dead_letter_config on the Lambda target. We'll come back to that.

Scheduled rules replace cron servers

EventBridge rules don't need to match incoming events. You can create rules that fire on a schedule using cron or rate expressions. This replaces the need for separate cron servers or scheduled EC2 instances.

A common use case: cleanup jobs, reconciliation tasks, or polling external systems. For example, reconciling orders every 5 minutes:

resource "aws_cloudwatch_event_rule" "reconcile_orders_schedule" {
  name                = "reconcile-orders-every-5m"
  schedule_expression = "rate(5 minutes)"
}

resource "aws_cloudwatch_event_target" "reconcile_orders_lambda" {
  rule      = aws_cloudwatch_event_rule.reconcile_orders_schedule.name
  arn       = aws_lambda_function.reconcile_orders.arn
  role_arn  = aws_iam_role.eventbridge_invoke_lambda.arn
  
  input = jsonencode({
    action = "reconcile"
    scope  = "all"
  })
}

resource "aws_lambda_permission" "allow_eventbridge_scheduled" {
  statement_id  = "AllowExecutionFromEventBridgeSchedule"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.reconcile_orders.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.reconcile_orders_schedule.arn
}

The input parameter lets you pass static data or transform incoming events. You can also use input_transformer for more complex JSON manipulations, though we usually keep it simple.

One gotcha: scheduled rules fire in UTC. If your team works in a different timezone and the schedule matters for operational windows, bake that into your documentation or use a wrapper Lambda that respects timezones.

Dead-letter queues for failed deliveries

Events fail. Your Lambda throws an exception, your SQS queue is misconfigured, or your Step Functions state machine hits an error state. By default, EventBridge retries, then silently drops the event. For critical events (payment confirmations, order placements), this is unacceptable.

Dead-letter queues (DLQs) capture failed events so you can investigate and replay them. Every target should have a DLQ configured:

resource "aws_sqs_queue" "lambda_dlq" {
  name                      = "payment-lambda-dlq"
  message_retention_seconds = 1209600  # 14 days
}

resource "aws_sqs_queue" "sqs_target_dlq" {
  name                      = "fulfillment-dlq"
  message_retention_seconds = 1209600
}

resource "aws_cloudwatch_event_target" "validate_payment_with_dlq" {
  rule      = aws_cloudwatch_event_rule.payment_processed.name
  arn       = aws_lambda_function.validate_payment.arn
  role_arn  = aws_iam_role.eventbridge_invoke_lambda.arn
  
  dead_letter_config {
    arn = aws_sqs_queue.lambda_dlq.arn
  }
}

resource "aws_cloudwatch_event_target" "fulfill_with_dlq" {
  rule      = aws_cloudwatch_event_rule.payment_processed.name
  arn       = aws_sqs_queue.fulfillment.arn
  role_arn  = aws_iam_role.eventbridge_send_sqs.arn
  
  dead_letter_config {
    arn = aws_sqs_queue.sqs_target_dlq.arn
  }
}

Set message retention to at least 14 days (AWS maximum) so failed events don't disappear before your on-call team sees them. Monitor DLQ depth with CloudWatch alarms; a growing DLQ is a signal that something's broken.

After investigation, you can manually replay events from the DLQ using the SendMessage API to re-inject them into your event bus. Build tooling for this if you replay frequently—it saves time and reduces human error.

Common mistakes

• Missing IAM roles: EventBridge needs permissions to invoke Lambda, write to SQS, etc. Define explicit roles for each target type and attach the right policies. A common error: sharing a single role across multiple targets, then being unable to audit who does what.
• No DLQs on critical targets: Not configuring DLQs means you lose visibility into failures. Always add them, especially for payment and order events.
• Overly broad event patterns: A pattern like {} (empty) matches everything. If you're testing and accidentally deploy this to production, you'll route all events through your target, potentially overwhelming it.
• Forgetting Lambda permissions: You configure the EventBridge target to invoke a Lambda, but forget to grant the lambda:InvokeFunction permission to EventBridge. The target silently fails.
• Event transformation in targets: Trying to extract and transform event data inside your Lambda instead of using EventBridge's input_transformer. This adds boilerplate and makes testing harder.
• No monitoring of rule matches: Set up CloudWatch metrics to track how many times each rule matches and how many times targets are invoked. Missing this means you won't notice when a rule stops matching.

Key takeaways

• Event patterns use JSON matching on source, detail-type, and nested detail fields. Use jsonencode() in Terraform to avoid syntax errors.
• One rule can have multiple targets (Lambda, SQS, Step Functions). Targets are invoked in parallel, so design for idempotence.
• Scheduled rules with rate or cron expressions eliminate the need for separate cron infrastructure. Remember they operate in UTC.
• Always configure dead-letter queues on targets. Monitor DLQ depth and build replay tooling for failed events.
• Grant explicit IAM permissions to EventBridge for each target type. Shared roles hide permission issues and complicate debugging.
• Test event patterns in the AWS console before deploying to production. Use CloudWatch Logs to verify rules are matching as expected.