Scaling ECS: From Prototype to Production

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that makes it easy to deploy, manage, and scale containerized applications. While spinning up a quick ECS prototype is straightforward, moving that same workload into production introduces a host of concerns: high availability, autoscaling, security, observability, and cost control. This tutorial walks through the journey of taking an ECS-based application from a rough prototype to a robust, production-ready deployment.

What Is ECS and Why Scaling Matters

ECS integrates deeply with the rest of the AWS ecosystem, supporting two launch types: EC2 (where you manage the underlying instances) and Fargate (where AWS manages the compute layer for you). In a prototype phase, you might run a single task behind a load balancer and call it done. In production, however, you need to handle traffic spikes, recover from failures, distribute work across Availability Zones, and keep costs predictable. Scaling ECS is not just about adding more tasks — it is about architecting the entire system so that growth happens gracefully.

From Prototype to Production: The Key Gaps

A typical prototype ECS setup suffers from several weaknesses that must be addressed before production:

Single Availability Zone deployment, creating a single point of failure.
Manual task definitions with hardcoded values and secrets.
No autoscaling, meaning traffic spikes cause latency or outages.
No health checks or graceful shutdown handling.
Limited logging and no metrics or alerting.
Over-provisioned resources, leading to wasted spend.

Each of these gaps maps directly to a production hardening step we will cover below.

Designing a Production Task Definition

The task definition is the blueprint for your containers. In production, it should be parameterized, secure, and resource-aware. Below is an example of a production-grade task definition written in Terraform.

resource "aws_ecs_task_definition" "app" {
  family                   = "production-app"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  cpu                      = "512"
  memory                   = "1024"
  execution_role_arn       = aws_iam_role.ecs_execution.arn
  task_role_arn            = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([
    {
      name      = "app"
      image     = "${var.ecr_repo_url}:latest"
      essential = true

      portMappings = [
        {
          containerPort = 8080
          protocol      = "tcp"
        }
      ]

      environment = [
        { name = "LOG_LEVEL", value = "info" },
        { name = "ENV",       value = "production" }
      ]

      secrets = [
        {
          name      = "DATABASE_URL"
          valueFrom = aws_secretsmanager_secret.db_url.arn
        }
      ]

      logConfiguration = {
        logDriver = "awslogs"
        options = {
          awslogs-group         = aws_cloudwatch_log_group.app.name
          awslogs-region        = var.region
          awslogs-stream-prefix = "prod"
        }
      }

      healthCheck = {
        command     = ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"]
        interval    = 30
        timeout     = 5
        retries     = 3
        startPeriod = 60
      }
    }
  ])
}

Notice several production-oriented choices: secrets are pulled from AWS Secrets Manager rather than embedded, a health check is defined so ECS can detect unhealthy containers, and structured logging is routed to CloudWatch. The CPU and memory values are intentionally modest — right-sizing is a best practice we will revisit.

Creating a Highly Available Service

An ECS service keeps the desired number of tasks running and restarts failed ones. For production, the service should span multiple Availability Zones and sit behind an Application Load Balancer.

resource "aws_ecs_service" "app" {
  name            = "production-app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 3
  launch_type     = "FARGATE"

  deployment_maximum_percent         = 200
  deployment_minimum_healthy_percent = 100

  network_configuration {
    subnets          = aws_subnet.private[*].id
    security_groups  = [aws_security_group.ecs_tasks.id]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.app.arn
    container_name   = "app"
    container_port   = 8080
  }

  deployment_circuit_breaker {
    enable   = true
    rollback = true
  }

  lifecycle {
    ignore_changes = [desired_count]
  }
}

The desired_count of 3 ensures that at least one task survives a single AZ failure when combined with subnets in three AZs. The deployment circuit breaker with rollback enabled prevents a bad deployment from taking down the service — if a new task version fails to stabilize, ECS automatically rolls back to the previous version. The ignore_changes directive on desired_count is important: it lets the autoscaling service manage the count without Terraform fighting it on every apply.

Configuring Autoscaling

Static task counts do not scale. ECS integrates with Application Autoscaling to adjust the desired count based on metrics. The most common approach is target tracking on average CPU utilization, but you can also use custom CloudWatch metrics such as request count per target.

resource "aws_appautoscaling_target" "app" {
  max_capacity       = 20
  min_capacity       = 3
  resource_id        = "service/${aws_ecs_cluster.main.name}/${aws_ecs_service.app.name}"
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"
}

resource "aws_appautoscaling_policy" "cpu" {
  name               = "cpu-target-tracking"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.app.resource_id
  scalable_dimension = aws_appautoscaling_target.app.scalable_dimension
  service_namespace  = aws_appautoscaling_target.app.service_namespace

  target_tracking_scaling_policy_configuration {
    target_value       = 65
    disable_scale_in   = false
    scale_in_cooldown  = 300
    scale_out_cooldown = 60

    predefined_metric_specification {
      predefined_metric_type = "ECSServiceAverageCPUUtilization"
    }
  }
}

Here the service scales out when average CPU exceeds 65%, with a fast scale-out cooldown of 60 seconds to respond quickly to spikes, and a slower 300-second scale-in cooldown to avoid flapping. The minimum of 3 maintains high availability, while the maximum of 20 caps cost. For request-driven workloads, consider adding a second policy based on ALBRequestCountPerTarget, which often reacts faster than CPU for bursty web traffic.

Handling Graceful Shutdowns

When ECS stops a task — whether for autoscaling, deployment, or failure — it sends a SIGTERM to the container and waits for the stop timeout (default 30 seconds) before SIGKILL. Your application must listen for this signal and shut down cleanly: stop accepting new requests, finish in-flight work, and close database connections. Here is a minimal Node.js example.

const http = require('http');

let server = http.createServer((req, res) => {
  if (req.url === '/health') {
    res.writeHead(200);
    return res.end('ok');
  }
  res.writeHead(200);
  res.end('hello');
});

server.listen(8080);

let shuttingDown = false;

function shutdown() {
  if (shuttingDown) return;
  shuttingDown = true;
  console.log('Received shutdown signal, draining...');
  server.close(() => {
    console.log('Connections closed, exiting.');
    process.exit(0);
  });

  // Force exit after 25 seconds if connections hang
  setTimeout(() => process.exit(1), 25000).unref();
}

process.on('SIGTERM', shutdown);
process.on('SIGINT', shutdown);

Pair this with a deregistration delay on your load balancer target group. Set the target group deregistration delay to around 30 seconds so that the load balancer stops sending new requests to a draining target while existing requests complete. This combination prevents dropped connections during deployments and scale-in events.

Observability and Alerting

Production systems need three pillars of observability: logs, metrics, and traces. ECS with Fargate sends stdout/stderr to CloudWatch Logs automatically when configured in the task definition. For metrics, enable CloudWatch Container Insights on the cluster.

resource "aws_ecs_cluster" "main" {
  name = "production-cluster"

  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}

With Container Insights, you get per-task and per-service metrics like CPU, memory, and network without instrumenting your code. Build alarms on the metrics that matter to users, not just infrastructure. For example, alarm on ALB 5xx error rate and target response time p99. A simple CloudWatch alarm for high error rate looks like this:

resource "aws_cloudwatch_metric_alarm" "high_5xx" {
  alarm_name          = "ecs-app-high-5xx"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  metric_name         = "HTTPCode_Target_5XX_Count"
  namespace           = "AWS/ApplicationELB"
  period              = 60
  statistic           = "Sum"
  threshold           = 10
  alarm_description   = "Alerts when 5xx errors exceed threshold"
  alarm_actions       = [aws_sns_topic.alerts.arn]

  dimensions = {
    LoadBalancer = aws_lb.main.arn_suffix
    TargetGroup  = aws_lb_target_group.app.arn_suffix
  }
}

For distributed tracing, integrate the AWS Distro for OpenTelemetry sidecar container into your task definition. This gives you end-to-end visibility across services without changing your application code significantly.

Security Hardening

Security is a production requirement, not an afterthought. Apply the following practices:

Use Fargate to avoid managing host instances and their patching.
Place tasks in private subnets with no public IP; route outbound traffic through a NAT gateway.
Use least-privilege IAM roles for both the task execution role and the task role.
Store all secrets in Secrets Manager or Parameter Store, never in the image or environment in plaintext.
Enable scan-on-push for your ECR repositories and block deployments of images with critical vulnerabilities.
Restrict security group rules to only the ports and sources required.

A minimal least-privilege task role might look like this:

data "aws_iam_policy_document" "task_assume" {
  statement {
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["ecs-tasks.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "ecs_task" {
  name               = "ecs-task-role"
  assume_role_policy = data.aws_iam_policy_document.task_assume.json
}

resource "aws_iam_role_policy" "task_s3" {
  name = "task-s3-access"
  role = aws_iam_role.ecs_task.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject"
        ]
        Resource = "${aws_s3_bucket.assets.arn}/*"
      }
    ]
  })
}

Cost Optimization

Scaling efficiently also means scaling economically. Right-size your tasks by observing actual CPU and memory usage through Container Insights over a representative period. Many teams over-provision memory because ECS requires explicit memory limits; instead, start small and adjust based on data. If you use the EC2 launch type, consider capacity providers with managed scaling to pack tasks efficiently and avoid paying for idle instances. With Fargate, take advantage of Fargate Spot for fault-tolerant workloads to reduce cost by up to 70%. Configure a capacity provider strategy that mixes on-demand and Spot capacity.

resource "aws_ecs_capacity_provider" "spot" {
  name = "fargate-spot"

  capacity_provider_configuration {
    capacity_provider = "FARGATE_SPOT"
    weight            = 1
  }
}

Deployment Strategies

For production, rolling deployments with circuit breakers are the baseline. For zero-downtime and the ability to test new versions on a subset of traffic, use ECS deployment with CodeDeploy for blue/green deployments. This integrates with the Application Load Balancer to shift traffic gradually from the old task set to the new one, and can automatically roll back if CloudWatch alarms fire during the deployment window.

resource "aws_codedeploy_app" "ecs" {
  name             = "production-app"
  compute_platform = "ECS"
}

resource "aws_codedeploy_deployment_group" "ecs" {
  app_name               = aws_codedeploy_app.ecs.name
  deployment_group_name  = "production-app-dg"
  service_role_arn       = aws_iam_role.codedeploy.arn
  deployment_config_name = "CodeDeployDefault.ECSAllAtOnce"

  auto_rollback_configuration {
    enabled = true
    events  = ["DEPLOYMENT_FAILURE"]
  }

  blue_green_deployment_config {
    deployment_ready_option {
      action_on_timeout = "CONTINUE_DEPLOYMENT"
    }

    terminate_blue_instances_on_deployment_success {
      action                           = "TERMINATE"
      termination_wait_time_in_minutes = 5
    }
  }

  ecs_service {
    cluster_name = aws_ecs_cluster.main.name
    service_name = aws_ecs_service.app.name
  }
}

Best Practices Summary

Always deploy across at least three Availability Zones with a minimum of three tasks.
Use target tracking autoscaling with both CPU and request-based metrics for web services.
Implement graceful shutdown in every container and tune the load balancer deregistration delay.
Enable Container Insights and alarm on user-facing metrics, not just CPU.
Store secrets in Secrets Manager and reference them in task definitions.
Right-size tasks using real usage data and consider Fargate Spot for cost savings.
Use blue/green deployments with automatic rollback for critical services.
Keep the desired count out of Terraform state when autoscaling manages it.
Enable ECR scan-on-push and fail builds on critical vulnerabilities.
Document your runbooks and test failure scenarios regularly, including AZ outages and deployment rollbacks.

Conclusion

Scaling ECS from a prototype to a production system is less about any single feature and more about combining many small, deliberate decisions: parameterized task definitions, multi-AZ services, responsive autoscaling, graceful shutdowns, comprehensive observability, least-privilege security, and cost-aware capacity choices. By addressing each of these areas systematically, you transform a fragile demo into a resilient platform that can absorb traffic growth, survive failures, and remain economical as demand evolves. The journey never truly ends — production systems require ongoing tuning, but starting from this foundation gives you the confidence to scale without fear.