Set log retention: what every AWS developer should know

Set log retention—a crucial cost control you're probably missing

CloudWatch logs are cheap at first glance, but unused retention policies quietly inflate your AWS bill month after month. I've seen teams shocked to discover logs accumulating for years, turning a $200/month logging expense into $2,000+. The fix is simple: set log retention to match your actual operational needs, not AWS's forever default.

Default retention is forever—and that costs money

By default, CloudWatch log groups have no expiration. Logs stream in from your Lambda functions, containers, and application servers, and they stay there indefinitely unless you explicitly delete them or set a retention policy.

Here's the cost model: CloudWatch charges roughly $0.50 per GB ingested and $0.03 per GB stored per month. A moderately active application might ingest 10 GB per day. That's 300 GB per month. After one year, you're storing 3.6 TB. At month 13, you're paying $108 just for storage—while still ingesting new logs. By year three, you're paying $1,080 per month for storage alone, plus ingestion costs.

The default "never expire" behavior was designed for compliance workloads that genuinely need historical logs. But most applications don't. Setting explicit retention policies forces you to be intentional about what you keep and why.

I learned this the hard way on a legacy trading platform. We had log groups from pilot services, failed deployments, and deprecated microservices all still shipping logs. One log group was collecting 500 MB per day but hadn't been read in two years. Once we audited and applied retention, our CloudWatch bill dropped 65%.

Set 14–30 days for most app logs

For typical applications, a 14- to 30-day retention window covers your operational needs without excessive storage costs. This window is long enough to investigate production issues, correlate events across multiple requests, and catch intermittent bugs. Beyond 30 days, you're almost always reaching into archives or compliance systems, not live troubleshooting.

Here's a practical breakdown:

• 14 days: Lean startups, high-velocity development teams, non-critical services. You still catch most issues during the active development cycle.
• 30 days: Standard production applications, SaaS platforms, e-commerce. Long enough for post-incident reviews and customer support investigations.
• 7 days: Extremely high-volume services where storage is the bottleneck and you have robust alerting.

Set retention via the AWS Console, CLI, or infrastructure-as-code:

aws logs put-retention-policy \
  --log-group-name /aws/lambda/my-function \
  --retention-in-days 30

If you're using CloudFormation or Terraform, embed retention directly in your log group definition:

resource "aws_cloudwatch_log_group" "app_logs" {
  name              = "/aws/ecs/my-app"
  retention_in_days = 30
}

Or in CDK (my preferred approach for complex stacks):

const logGroup = new logs.LogGroup(this, 'AppLogs', {
  logGroupName: '/aws/ecs/my-app',
  retention: logs.RetentionDays.ONE_MONTH,
});

The key is making retention a first-class decision in your infrastructure code, not an afterthought. When you provision a log group, you decide its lifetime immediately.

Archive to S3 for compliance

Retention policies delete logs after the window expires. But compliance regulations (SOC 2, HIPAA, PCI-DSS) often require you to keep logs for years. The solution: export to S3 before deletion and archive with intelligent tiering.

CloudWatch Logs Insights and the API make this straightforward. You have two patterns:

Pattern 1: Manual export before deletion

Export logs to S3 on a schedule (usually weekly or monthly), then let them expire from CloudWatch:

aws logs create-export-task \
  --log-group-name /aws/lambda/my-function \
  --from 1609459200000 \
  --to 1609545600000 \
  --destination my-log-archive-bucket \
  --destination-prefix logs/my-function/2021-01-01

Run this via Lambda on a schedule (EventBridge rule) to archive older logs automatically.

Pattern 2: Log Filter Subscription

For critical logs, stream them continuously to Kinesis or Firehose, which auto-archives to S3:

aws logs put-subscription-filter \
  --log-group-name /aws/lambda/critical-function \
  --filter-name "critical-to-s3" \
  --filter-pattern "[...]" \
  --destination-arn arn:aws:firehose:region:account:deliverystream/log-archive

Firehose compresses and batches logs into S3 objects, keeping hot logs in CloudWatch but maintaining a cheap, queryable archive in S3. Then configure S3 Intelligent-Tiering to move old objects to Glacier automatically after 30 days.

On a recent CRM platform, we used this hybrid: 14-day retention in CloudWatch for operational logs, continuous Firehose export to S3 with Glacier archival after 90 days. Storage cost dropped 80%, and we still had full compliance history.

Filter noisy debug logs

High-verbosity debug logging is a silent cost killer. A single misconfigured logger pushing DEBUG-level events from a busy service can generate gigabytes per day of noise.

Three practical controls:

1. Filter at the application level

Your logging library (Serilog, winston, pino) should respect environment-based log levels:

// Node.js with pino
const logger = require('pino')({
  level: process.env.LOG_LEVEL || 'info', // 'debug' only in dev
});

logger.debug('expensive database query details'); // won't ship in prod

This is the cheapest filter—logs never leave your process.

2. Filter at the source with subscription filters

If you can't change application code immediately, CloudWatch subscription filters drop unwanted logs before they're stored:

aws logs put-subscription-filter \
  --log-group-name /aws/lambda/my-function \
  --filter-name "drop-debug" \
  --filter-pattern "[timestamp, request_id, level != ERROR && level != WARN, ...]" \
  --destination-arn arn:aws:lambda:region:account:function:filter-logs

The filter expression matches on log structure; non-matching logs are dropped. You still pay for ingestion, but not storage.

3. Sample noisy logs

For intentional high-volume streams (e.g., health checks, telemetry), sample them at the application level:

// Log only 1 in 100 requests
if (Math.random() < 0.01) {
  logger.info('health check', { endpoint, responseTime });
}

Or use probabilistic sampling in your instrumentation library (OpenTelemetry supports this out of the box).

I once discovered a microservice logging the entire request body on every call—including large JSON payloads. One client's integration was sending 5 MB requests every 10 seconds. That single noisy log was costing $500/month. Filtering it took 10 minutes and saved the team $6,000 per year.

Common mistakes

• Setting retention too long "just in case": I've seen teams set 1 year retention on every log group, then never actually access logs older than 30 days. You can always export to S3 for long-term storage; don't pay CloudWatch premium pricing for rarely-touched data.
• Forgetting to set retention on new log groups: Default is still "forever." Automate this with infrastructure-as-code or CloudFormation templates. Never manually create log groups in the console without immediately setting retention.
• Ignoring temporary or test log groups: A log group from a failed experiment or feature branch can quietly accumulate logs for months. Audit and delete unused log groups quarterly.
• Mixing hot and cold retention: Don't export and archive while also keeping logs in CloudWatch forever. Pick a retention window, then export older data, and delete from CloudWatch.
• Exporting without compression or deduplication: S3 export tasks create one object per export. Enable gzip compression and use S3 Intelligent-Tiering to avoid paying for redundant storage.

Key takeaways

• CloudWatch's default "never expire" retention silently inflates costs. Always set explicit retention policies in your infrastructure code.
• 14–30 days covers most operational use cases. Anything older belongs in S3 archives, not hot CloudWatch storage.
• For compliance, use Firehose or export tasks to move logs to S3 before deletion, then tier to Glacier for long-term retention.
• Filter noisy debug logs at the application level first (cheapest), then with subscription filters if needed.
• Audit log groups quarterly to catch unused or misconfigured streams that are still generating charges.
• Make retention a first-class decision in infrastructure-as-code, not a manual afterthought.