Terraform performance tips: patterns, pitfalls and fixes

Terraform performance tips: patterns, pitfalls and fixes

We've all watched a terraform plan hang for ten minutes while we stare at the terminal, wondering if something broke. In production environments with hundreds or thousands of resources, slow Terraform runs become a real bottleneck—blocking deployments, frustrating teams, and eating CI/CD minutes. The good news is that most performance problems stem from a handful of predictable patterns, and fixing them can cut your plan times from minutes to seconds.

This guide walks through the key Terraform performance tips that have helped us optimize infrastructure code across large e-commerce, SaaS, and ride-hailing deployments. We'll cover state management, parallelism tuning, data source abuse, and CI provider caching—all with real scenarios and code you can apply immediately.

Understanding where Terraform spends time

Before jumping into fixes, it helps to know what's actually slow. Terraform's runtime breaks down into three main phases:

• Init: downloading and locking providers and modules (one-time per directory change)
• Plan: reading state, querying remote APIs, building the execution graph
• Apply: executing changes sequentially or in parallel where dependencies allow

Most slowdowns happen during plan, where Terraform must refresh state by calling provider APIs. Every data source, every AWS API call, every database query adds latency. State size also matters—larger state files take longer to parse and process, especially when you're working with hundreds of managed resources.

Smaller states: splitting and targeting

A monolithic state file containing your entire infrastructure—networks, databases, applications, monitoring—becomes a performance drag. Each plan operation must refresh the entire graph, even if you're only changing a single application stack.

We've seen teams manage this with two proven strategies:

Root modules per logical domain

Instead of one terraform/main.tf handling everything, organize by responsibility:

terraform/
├── networking/
│   ├── main.tf
│   ├── vpc.tf
│   └── state.tfvars
├── databases/
│   ├── main.tf
│   ├── rds.tf
│   └── state.tfvars
└── applications/
    ├── main.tf
    ├── eks.tf
    └── state.tfvars

Each directory has its own state file and can be planned independently. A developer changing an EKS configuration won't trigger a refresh of your entire RDS layer. In one e-commerce platform we worked on, splitting a 2,500-resource state into five focused states cut average plan time from 8 minutes to 90 seconds per domain.

Using state targeting strategically

When you must work in a large state, use -target and terraform_state data source filtering to scope operations:

terraform plan -target=aws_eks_cluster.main -target=aws_eks_node_group.workers

This tells Terraform to only evaluate and refresh those resources and their dependencies, skipping unrelated state queries. Be cautious though—targeting changes the meaning of your plan. Always run a full plan before applying to ensure you haven't missed side effects.

Parallelism: the -parallelism flag and its limits

By default, Terraform applies and refreshes resources with a parallelism of 10—meaning it processes up to 10 resources concurrently. You can increase this:

terraform plan -parallelism=30

In our SaaS platform with 200+ independent Lambda functions, bumping parallelism from 10 to 30 reduced apply times from 12 minutes to 4 minutes. The gain comes from resource independence—if most resources have no inter-dependencies, higher parallelism means more API calls happening simultaneously.

When higher parallelism backfires

Don't assume higher always means faster. We've hit rate limits on AWS API calls when pushing parallelism to 50+ in large deployments. The provider would throttle, and Terraform would retry, negating any speed gain and adding jitter. Start at 20–30, monitor apply duration and error rates, and adjust based on your cloud provider's quotas.

Also check your CI environment. If you're running in a shared GitHub Actions runner or small EC2 instance, higher parallelism may cause memory pressure or CPU saturation, making things slower overall.

Avoid unnecessary data sources

Data sources are convenience—they let you query existing infrastructure without managing it in state. But every data source is an API call, and API calls add latency. Many performance issues we've diagnosed came down to gratuitous data source usage.

Real-world scenario: the redundant VPC lookup

A common pattern we see:

# ❌ SLOW: querying VPC every plan, even though we control it
data "aws_vpc" "main" {
  filter {
    name   = "tag:Name"
    values = ["production-vpc"]
  }
}

resource "aws_security_group" "app" {
  vpc_id = data.aws_vpc.main.id
  name   = "app"
}

If you created the VPC in the same Terraform workspace, reference it directly:

# ✅ FAST: use resource output
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "production-vpc"
  }
}

resource "aws_security_group" "app" {
  vpc_id = aws_vpc.main.id
  name   = "app"
}

Removing data sources you control saves seconds per plan. In one ride-hailing infrastructure refactor, we eliminated 40+ unnecessary data sources looking up subnets, security groups, and IAM roles we owned—plan time dropped from 5 minutes to 90 seconds.

When data sources are justified

Keep data sources for resources managed outside Terraform: shared AWS accounts, third-party services, or legacy infrastructure. Example:

# ✅ APPROPRIATE: querying a shared logging account's S3 bucket
data "aws_s3_bucket" "central_logs" {
  bucket = "org-central-logs"
}

resource "aws_s3_bucket_policy" "app_logs" {
  bucket = data.aws_s3_bucket.central_logs.id
  policy = jsonencode({...})
}

Also batch lookups where you can. Instead of 10 separate data sources, use a single data "aws_instances" with filters and index the results.

Cache providers in CI

Provider initialization happens during terraform init. Each run downloads provider binaries from the Terraform Registry, validates checksums, and installs them. On slow networks or with many providers, this adds 30–60 seconds per CI job.

Docker layer caching

If you run Terraform in Docker, cache the provider directory between builds:

# Dockerfile
FROM hashicorp/terraform:1.7

WORKDIR /terraform

# Copy lock file first (changes rarely)
COPY .terraform.lock.hcl .

# Restore provider cache from layer
RUN terraform providers lock -platform=linux_amd64

COPY . .

RUN terraform init -backend=false

Layer caching means subsequent builds reuse the provider layer if the lock file hasn't changed, shaving 30 seconds off every CI run.

CI-native provider caching

Most CI systems support artifact caching. For GitHub Actions:

- name: Cache Terraform
  uses: actions/cache@v3
  with:
    path: |
      ~/.terraform.d/plugin-cache
      .terraform
    key: tf-${{ hashFiles('**/.terraform.lock.hcl') }}
    restore-keys: tf-

- name: Terraform Init
  run: terraform init

The first build warms the cache; subsequent builds (same lock file) skip provider downloads. Combined with Docker layer caching, we've cut CI init times from 90 seconds to under 10 seconds in large deployments.

Common mistakes

• Querying AWS for every resource attribute: Using data.aws_ami to fetch the latest AMI on every plan when you could reference a Terraform-managed AMI or lock it in a variable. Hundreds of small API calls add up.
• Blocking dependencies you don't need: Setting depends_on = [aws_lambda_function.processor] when there's no actual dependency forces sequential apply. Remove artificial constraints and let Terraform parallelize.
• Ignoring state locking in CI: Running concurrent plans on the same state without locking causes state corruption. Always enable DynamoDB state locking for shared environments.
• Not using .terraform.lock.hcl: Terraform recalculates provider versions if the lock file is missing, adding network I/O. Commit the lock file to git and regenerate only when intentional.
• Over-targeting in production: Using -target to skip infrastructure changes is tempting but risky. It's easy to create resource drift or broken references. Use it for debugging, not deployments.

Key takeaways

• Split large monolithic states into smaller, domain-focused root modules to reduce refresh scope per operation.
• Start parallelism at 20–30 and test whether your cloud provider's rate limits allow higher values without throttling.
• Eliminate data sources for infrastructure you control; keep them only for external or shared resources.
• Cache provider binaries in Docker layers and CI artifacts to avoid repeated registry downloads.
• Monitor terraform plan duration in CI and use terraform show -json | jq '.resource_drift | length' to spot unnecessary refresh operations.
• Commit .terraform.lock.hcl to version control and regenerate only when deliberately updating providers.