Introduction to Scaling Fargate
AWS Fargate is a serverless compute engine for containers that works with both Amazon Elastic Container Service (ECS) and Amazon Elastic Kubernetes Service (EKS). When you build a prototype on Fargate, you typically focus on getting your application running quickly. However, moving to production requires careful consideration of scaling strategies, resource allocation, monitoring, and cost optimization. This tutorial walks you through the journey of scaling a Fargate-based application from a simple prototype to a robust, production-ready system.
What Is Fargate Scaling?
Fargate scaling refers to the ability to automatically adjust the number of running container instances (tasks) based on demand. Unlike traditional EC2-based deployments where you manage the underlying infrastructure, Fargate abstracts away the server management, allowing you to focus on container-level scaling. Scaling on Fargate involves two primary mechanisms: Service Auto Scaling, which adjusts the number of tasks running, and Application Auto Scaling, which defines the policies and metrics that trigger scaling actions.
Each Fargate task is allocated a specific combination of CPU and memory resources. When demand increases, Auto Scaling launches additional tasks to distribute the load. When demand decreases, it terminates unnecessary tasks to reduce costs. This elasticity is what makes Fargate particularly attractive for production workloads with variable traffic patterns.
Why Scaling Matters
Scaling is not just about handling more traffic — it is about maintaining performance, reliability, and cost efficiency simultaneously. A prototype might run fine with a single task, but production environments face unpredictable traffic spikes, regional outages, and varying workload patterns. Without proper scaling, your application could become unresponsive during peak loads or waste significant money during quiet periods.
Key reasons scaling matters include:
- Performance: Maintaining low latency and fast response times under varying load conditions.
- Availability: Ensuring enough tasks are running to handle traffic even if some fail or become unhealthy.
- Cost optimization: Running only the resources you need, when you need them.
- Resilience: Automatically recovering from failures by replacing unhealthy tasks and scaling to meet demand.
- Operational efficiency: Reducing manual intervention for capacity planning and traffic management.
Setting Up Your Fargate Service
Before you can scale, you need a properly configured Fargate service. Let us start with a CloudFormation template that defines a basic ECS Fargate service. This template creates a task definition, an ECS service, and the necessary IAM roles.
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Fargate Service for Production'
Parameters:
EnvironmentName:
Type: String
Default: production
Description: 'Environment name'
ContainerImage:
Type: String
Description: 'Container image URI'
TaskCPU:
Type: String
Default: '512'
AllowedValues: ['256', '512', '1024', '2048', '4096']
TaskMemory:
Type: String
Default: '1024'
Description: 'Memory in MB'
DesiredCount:
Type: Number
Default: 2
Resources:
TaskExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
TaskRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: FargateTaskAccess
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub '${EnvironmentName}-app'
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: !Ref TaskCPU
Memory: !Ref TaskMemory
ExecutionRoleArn: !GetAtt TaskExecutionRole.Arn
TaskRoleArn: !GetAtt TaskRole.Arn
ContainerDefinitions:
- Name: app
Image: !Ref ContainerImage
PortMappings:
- ContainerPort: 8080
Protocol: tcp
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref LogGroup
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: ecs
HealthCheck:
Command:
- CMD-SHELL
- curl -f http://localhost:8080/health || exit 1
Interval: 30
Timeout: 5
Retries: 3
StartPeriod: 60
LogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub '/ecs/${EnvironmentName}-app'
RetentionInDays: 30
ECSService:
Type: AWS::ECS::Service
Properties:
ServiceName: !Sub '${EnvironmentName}-app-service'
Cluster: !Ref ECSCluster
TaskDefinition: !Ref TaskDefinition
DesiredCount: !Ref DesiredCount
LaunchType: FARGATE
HealthCheckGracePeriodSeconds: 60
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
Subnets:
- !Ref PrivateSubnetA
- !Ref PrivateSubnetB
SecurityGroups:
- !Ref ServiceSecurityGroup
LoadBalancers:
- ContainerName: app
ContainerPort: 8080
TargetGroupArn: !Ref TargetGroup
This template sets up the foundation. Notice the health check configuration in the container definition — this is critical for production because it allows ECS to detect and replace unhealthy tasks automatically.
Configuring Auto Scaling
Now that you have a running service, the next step is to configure Auto Scaling. AWS Fargate supports several scaling strategies, but the most common approach for production is target tracking scaling. This strategy adjusts the number of tasks to maintain a specific metric at a target value, such as keeping average CPU utilization at 50%.
Target Tracking Scaling Policy
Target tracking is the simplest and most effective scaling strategy for most applications. You define a target value for a metric, and AWS automatically scales your service up or down to maintain that target. Here is how to configure it using CloudFormation:
ScalableTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
Properties:
MaxCapacity: 20
MinCapacity: 2
ResourceId: !Sub 'service/${ECSCluster}/${ECSService.Name}'
ScalableDimension: ecs:service:DesiredCount
ServiceNamespace: ecs
CPUTrackingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${EnvironmentName}-cpu-tracking'
PolicyType: TargetTrackingScaling
ScalingTargetId: !Ref ScalableTarget
TargetTrackingScalingPolicyConfiguration:
TargetValue: 50.0
PredefinedMetricSpecification:
PredefinedMetricType: ECSServiceAverageCPUUtilization
ScaleInCooldown: 300
ScaleOutCooldown: 60
MemoryTrackingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${EnvironmentName}-memory-tracking'
PolicyType: TargetTrackingScaling
ScalingTargetId: !Ref ScalableTarget
TargetTrackingScalingPolicyConfiguration:
TargetValue: 70.0
PredefinedMetricSpecification:
PredefinedMetricType: ECSServiceAverageMemoryUtilization
ScaleInCooldown: 300
ScaleOutCooldown: 60
In this configuration, the service will scale out when average CPU utilization exceeds 50% or memory utilization exceeds 70%. The ScaleOutCooldown of 60 seconds allows rapid response to traffic spikes, while the ScaleInCooldown of 300 seconds prevents premature scale-in during brief dips in traffic.
Step Scaling Policy
For more granular control, you can use step scaling. This approach allows you to define different scaling adjustments based on the magnitude of the metric breach. For example, you might add 2 tasks when CPU is between 50% and 70%, but add 5 tasks when CPU exceeds 70%.
CloudWatchAlarmHighCPU:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${EnvironmentName}-high-cpu'
MetricName: CPUUtilization
Namespace: AWS/ECS
Statistic: Average
Period: 60
EvaluationPeriods: 2
Threshold: 50
ComparisonOperator: GreaterThanThreshold
Dimensions:
- Name: ServiceName
Value: !GetAtt ECSService.Name
- Name: ClusterName
Value: !Ref ECSCluster
StepScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${EnvironmentName}-step-scaling'
PolicyType: StepScaling
ScalingTargetId: !Ref ScalableTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 60
StepAdjustments:
- MetricIntervalLowerBound: 0
MetricIntervalUpperBound: 20
ScalingAdjustment: 2
- MetricIntervalLowerBound: 20
MetricIntervalUpperBound: 40
ScalingAdjustment: 4
- MetricIntervalLowerBound: 40
ScalingAdjustment: 6
Custom Metrics for Smarter Scaling
CPU and memory utilization are useful metrics, but they do not always reflect the actual user experience. For example, your application might be CPU-light but I/O-heavy, or it might have a queue-based architecture where the number of pending messages is a better scaling indicator. Using custom metrics allows you to scale based on what actually matters for your application.
First, publish custom metrics from your application:
import boto3
import time
cloudwatch = boto3.client('cloudwatch')
def publish_metric(queue_depth, processing_time):
cloudwatch.put_metric_data(
Namespace='MyApp/Production',
MetricData=[
{
'MetricName': 'QueueDepth',
'Value': queue_depth,
'Unit': 'Count',
'Dimensions': [
{
'Name': 'Environment',
'Value': 'production'
}
]
},
{
'MetricName': 'ProcessingTime',
'Value': processing_time,
'Unit': 'Milliseconds',
'Dimensions': [
{
'Name': 'Environment',
'Value': 'production'
}
]
}
]
)
# Example usage in your application
def process_request():
start_time = time.time()
# Your application logic here
queue_depth = get_queue_depth()
processing_time = (time.time() - start_time) * 1000
publish_metric(queue_depth, processing_time)
Then, create a CloudWatch alarm and scaling policy based on the custom metric:
CustomMetricAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${EnvironmentName}-queue-depth-alarm'
Namespace: MyApp/Production
MetricName: QueueDepth
Statistic: Average
Period: 60
EvaluationPeriods: 1
Threshold: 100
ComparisonOperator: GreaterThanThreshold
Dimensions:
- Name: Environment
Value: production
CustomMetricScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: !Sub '${EnvironmentName}-queue-scaling'
PolicyType: StepScaling
ScalingTargetId: !Ref ScalableTarget
StepScalingPolicyConfiguration:
AdjustmentType: ChangeInCapacity
Cooldown: 120
StepAdjustments:
- MetricIntervalLowerBound: 0
ScalingAdjustment: 3
Alarms:
- !Ref CustomMetricAlarm
Load Balancer Configuration for Scaling
When your Fargate service scales, new tasks need to be registered with the load balancer automatically. The Application Load Balancer (ALB) integrates seamlessly with ECS, but you need to configure it correctly for production. Key settings include health check paths, deregistration delay, and stickiness.
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: !Sub '${EnvironmentName}-app-tg'
Port: 8080
Protocol: HTTP
VpcId: !Ref VpcId
TargetType: ip
DeregistrationDelay: 30
HealthCheckEnabled: true
HealthCheckPath: /health
HealthCheckIntervalSeconds: 15
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
UnhealthyThresholdCount: 3
Matcher:
HttpCode: '200'
TargetGroupAttributes:
- Key: stickiness.enabled
Value: 'true'
- Key: stickiness.type
Value: lb_cookie
- Key: stickiness.duration_seconds
Value: '3600'
- Key: deregistration_delay.timeout_seconds
Value: '30'
LoadBalancerListenerRule:
Type: AWS::ElasticLoadBalancingV2::ListenerRule
Properties:
ListenerArn: !Ref ListenerArn
Priority: 1
Conditions:
- Field: path-pattern
Values:
- /*
Actions:
- Type: forward
TargetGroupArn: !Ref TargetGroup
The DeregistrationDelay of 30 seconds ensures that in-flight requests complete before a task is fully deregistered during scale-in events. The health check interval of 15 seconds with a healthy threshold of 2 means a new task becomes available after approximately 30 seconds, which is fast enough for most production scenarios.
Monitoring and Observability
Scaling without monitoring is like driving blindfolded. You need comprehensive observability to understand how your application behaves under different load conditions. CloudWatch provides built-in metrics for ECS, but you should also collect application-level metrics, logs, and traces.
CloudWatch Dashboard
Create a CloudWatch dashboard to visualize your Fargate service health:
CloudWatchDashboard:
Type: AWS::CloudWatch::Dashboard
Properties:
DashboardName: !Sub '${EnvironmentName}-fargate-dashboard'
DashboardBody: !Sub |
{
"widgets": [
{
"type": "metric",
"x": 0, "y": 0, "width": 12, "height": 6,
"properties": {
"metrics": [
["AWS/ECS", "CPUUtilization", "ServiceName", "${ECSService.Name}", "ClusterName", "${ECSCluster}"],
["AWS/ECS", "MemoryUtilization", "ServiceName", "${ECSService.Name}", "ClusterName", "${ECSCluster}"]
],
"period": 60,
"stat": "Average",
"region": "${AWS::Region}",
"title": "Resource Utilization"
}
},
{
"type": "metric",
"x": 12, "y": 0, "width": 12, "height": 6,
"properties": {
"metrics": [
["AWS/ApplicationELB", "RequestCount", "LoadBalancer", "${LoadBalancerFullName}"],
["AWS/ApplicationELB", "TargetResponseTime", "LoadBalancer", "${LoadBalancerFullName}"],
["AWS/ApplicationELB", "HTTPCode_Target_5XX_Count", "LoadBalancer", "${LoadBalancerFullName}"]
],
"period": 60,
"stat": "Sum",
"region": "${AWS::Region}",
"title": "Traffic and Errors"
}
},
{
"type": "metric",
"x": 0, "y": 6, "width": 24, "height": 6,
"properties": {
"metrics": [
["AWS/ApplicationELB", "HealthyHostCount", "TargetGroup", "${TargetGroupFullName}", "LoadBalancer", "${LoadBalancerFullName}"],
["AWS/ApplicationELB", "UnHealthyHostCount", "TargetGroup", "${TargetGroupFullName}", "LoadBalancer", "${LoadBalancerFullName}"]
],
"period": 60,
"stat": "Average",
"region": "${AWS::Region}",
"title": "Task Health"
}
}
]
}
CloudWatch Alarms for Alerting
Set up alarms to alert your team when things go wrong:
HighCPUAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${EnvironmentName}-cpu-critical'
AlarmDescription: 'CPU utilization above 85% for 5 minutes'
Namespace: AWS/ECS
MetricName: CPUUtilization
Statistic: Average
Period: 60
EvaluationPeriods: 5
Threshold: 85
ComparisonOperator: GreaterThanThreshold
Dimensions:
- Name: ServiceName
Value: !GetAtt ECSService.Name
- Name: ClusterName
Value: !Ref ECSCluster
AlarmActions:
- !Ref SNSTopicArn
HighErrorRateAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${EnvironmentName}-error-rate-critical'
AlarmDescription: '5XX error rate above 5% for 3 minutes'
Namespace: AWS/ApplicationELB
MetricName: HTTPCode_Target_5XX_Count
Statistic: Sum
Period: 60
EvaluationPeriods: 3
Threshold: 10
ComparisonOperator: GreaterThanThreshold
Dimensions:
- Name: LoadBalancer
Value: !Ref LoadBalancerFullName
AlarmActions:
- !Ref SNSTopicArn
TaskReplacementAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Sub '${EnvironmentName}-task-replacement'
AlarmDescription: 'Multiple task replacements detected'
Namespace: AWS/ECS
MetricName: RunningTaskCount
Statistic: Minimum
Period: 300
EvaluationPeriods: 2
Threshold: 1
ComparisonOperator: LessThanThreshold
Dimensions:
- Name: ServiceName
Value: !GetAtt ECSService.Name
- Name: ClusterName
Value: !Ref ECSCluster
AlarmActions:
- !Ref SNSTopicArn
Best Practices for Production Fargate
Right-Size Your Tasks
One of the most common mistakes when moving from prototype to production is over-provisioning or under-provisioning task resources. Fargate charges based on the CPU and memory you allocate, so right-sizing directly impacts cost. Start with a baseline configuration, monitor utilization over a representative period, and adjust accordingly. A good rule of thumb is to target 50-70% average CPU utilization under normal load, which gives headroom for traffic spikes while keeping costs reasonable.
Use Multiple Availability Zones
Always distribute your Fargate tasks across at least two Availability Zones. This is configured in the network configuration of your ECS service. If one AZ experiences an outage, your application continues running in the other AZ. The CloudFormation template earlier in this tutorial already demonstrates this by specifying two private subnets in different AZs.
Implement Circuit Breakers and Timeouts
When your service scales, it interacts with other services that may also be under load. Implement circuit breakers to prevent cascading failures:
import asyncio
from functools import wraps
import time
class CircuitBreaker:
def __init__(self, failure_threshold=5, recovery_timeout=60):
self.failure_threshold = failure_threshold
self.recovery_timeout = recovery_timeout
self.failure_count = 0
self.last_failure_time = None
self.state = 'closed'
def __call__(self, func):
@wraps(func)
async def wrapper(*args, **kwargs):
if self.state == 'open':
if time.time() - self.last_failure_time > self.recovery_timeout:
self.state = 'half-open'
else:
raise Exception('Circuit breaker is open')
try:
result = await asyncio.wait_for(
func(*args, **kwargs),
timeout=5.0
)
if self.state == 'half-open':
self.state = 'closed'
self.failure_count = 0
return result
except Exception as e:
self.failure_count += 1
self.last_failure_time = time.time()
if self.failure_count >= self.failure_threshold:
self.state = 'open'
raise
return wrapper
# Usage
breaker = CircuitBreaker(failure_threshold=5, recovery_timeout=60)
@breaker
async def call_external_api(endpoint):
# Your API call logic here
pass
Configure Graceful Shutdown
When Fargate scales in or deploys a new version, it sends a SIGTERM signal to your container. Your application should handle this signal gracefully by stopping accepting new requests, completing in-flight work, and then exiting cleanly. ECS waits for the DeregistrationDelay before forcefully terminating the task.
import signal
import sys
import asyncio
from aiohttp import web
class GracefulServer:
def __init__(self):
self.shutdown_event = asyncio.Event()
self.active_requests = 0
self.lock = asyncio.Lock()
async def health_check(self, request):
if self.shutdown_event.is_set():
return web.json_response(
{'status': 'shutting_down'},
status=503
)
return web.json_response({'status': 'healthy'})
async def handle_request(self, request):
if self.shutdown_event.is_set():
return web.json_response(
{'error': 'server shutting down'},
status=503
)
async with self.lock:
self.active_requests += 1
try:
# Process request
await asyncio.sleep(0.1)
return web.json_response({'result': 'success'})
finally:
async with self.lock:
self.active_requests -= 1
async def shutdown(self, signum, frame):
print(f'Received signal {signum}, initiating graceful shutdown...')
self.shutdown_event.set()
# Wait for active requests to complete (max 30 seconds)
for _ in range(30):
if self.active_requests == 0:
break
print(f'Waiting for {self.active_requests} active requests...')
await asyncio.sleep(1)
print('Shutdown complete')
sys.exit(0)
app = web.Application()
server = GracefulServer()
app.router.add_get('/health', server.health_check)
app.router.add_post('/api', server.handle_request)
# Register signal handlers
signal.signal(signal.SIGTERM, lambda s, f: asyncio.create_task(server.shutdown(s, f)))
signal.signal(signal.SIGINT, lambda s, f: asyncio.create_task(server.shutdown(s, f)))
if __name__ == '__main__':
web.run_app(app, port=8080)
Use Scheduled Scaling for Predictable Patterns
If your traffic follows predictable patterns, such as higher load during business hours, use scheduled scaling to pre-provision capacity before traffic arrives:
ScheduledScaleOut:
Type: AWS::ApplicationAutoScaling::ScheduledAction
Properties:
ServiceNamespace: ecs
ResourceId: !Sub 'service/${ECSCluster}/${ECSService.Name}'
ScalableDimension: ecs:service:DesiredCount
ScalableTargetAction:
MinCapacity: 5
MaxCapacity: 30
Schedule: 'cron(0 8 ? * MON-FRI *)'
Timezone: America/New_York
ScheduledScaleIn:
Type: AWS::ApplicationAutoScaling::ScheduledAction
Properties:
ServiceNamespace: ecs
ResourceId: !Sub 'service/${ECSCluster}/${ECSService.Name}'
ScalableDimension: ecs:service:DesiredCount
ScalableTargetAction:
MinCapacity: 2
MaxCapacity: 10
Schedule: 'cron(0 18 ? * MON-FRI *)'
Timezone: America/New_York
Implement Blue-Green Deployments
For production deployments, use blue-green deployment to minimize downtime and reduce risk. ECS supports this through CodeDeploy integration:
CodeDeployApplication:
Type: AWS::CodeDeploy::Application
Properties:
ApplicationName: !Sub '${EnvironmentName}-app-deploy'
ComputePlatform: ECS
CodeDeployDeploymentGroup:
Type: AWS::CodeDeploy::DeploymentGroup
Properties:
ApplicationName: !Ref CodeDeployApplication
DeploymentGroupName: !Sub '${EnvironmentName}-app-dg'
ServiceRoleArn: !Ref CodeDeployServiceRoleArn
DeploymentConfig:
DeploymentConfigName: CodeDeployDefault.ECSAllAtOnce
DeploymentStyle:
DeploymentType: BLUE_GREEN
DeploymentOption: WITH_TRAFFIC_CONTROL
BlueGreenDeploymentConfiguration:
TerminateBlueInstancesOnDeploymentSuccess:
Action: TERMINATE
TerminationWaitTimeInMinutes: 15
DeploymentReadyOption:
ActionOnTimeout: CONTINUE_DEPLOYMENT
WaitTimeInMinutes: 30
AutoRollbackConfiguration:
Enabled: true
Events:
- DEPLOYMENT_FAILURE
- DEPLOYMENT_STOP_ON_ALARM
- DEPLOYMENT_STOP_ON_REQUEST
Optimize Container Image Size
Smaller container images start faster, which directly impacts your scaling responsiveness. When a scale-out event triggers, the new task must pull the container image before it can start serving traffic. Use multi-stage builds and minimal base images:
# Build stage
FROM python:3.11-slim as builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt
# Production stage
FROM python:3.11-slim
WORKDIR /app
# Copy only necessary files from builder
COPY --from=builder /root/.local /root/.local
COPY . .
# Ensure scripts in .local are usable
ENV PATH=/root/.local/bin:$PATH
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
# Run as non-root user
RUN useradd -m appuser
USER appuser
EXPOSE 8080
CMD ["python", "-m", "gunicorn", "--bind", "0.0.0.0:8080", "--workers", "2", "app:app"]
Cost Optimization Strategies
As you scale, costs can quickly spiral out of control. Here are several strategies to keep costs in check while maintaining performance:
- Use Fargate Spot for fault-tolerant workloads: Fargate Spot offers up to 70% discount compared to on-demand pricing. It is ideal for batch processing, queue workers, and other workloads that can tolerate interruptions.
- Implement aggressive scale-in policies: Set appropriate cooldown periods and lower thresholds for scale-in to avoid running unnecessary tasks.
- Use Savings Plans: If you have predictable, steady-state workloads, commit to Fargate usage with Savings Plans for up to 50% discount.
- Monitor and eliminate idle tasks: Use CloudWatch metrics to identify tasks with consistently low utilization and adjust your minimum capacity accordingly.
Here is how to configure a Fargate Spot capacity provider for cost optimization:
SpotCapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
Name: !Sub '${EnvironmentName}-fargate-spot'
Type: FARGATE_SPOT
AutoScalingGroupProvider:
AutoScalingGroupArn: !Ref AutoScalingGroup
ManagedScaling:
Status: ENABLED
TargetCapacity: 100
ManagedTerminationProtection: DISABLED
OnDemandCapacityProvider:
Type: AWS::ECS::CapacityProvider
Properties:
Name: !Sub '${EnvironmentName}-fargate-ondemand'
Type: FARGATE
ClusterCapacityProviderAssociations:
Type: AWS::ECS::ClusterCapacityProviderAssociations
Properties:
Cluster: !Ref ECSCluster
CapacityProviders:
- !Ref SpotCapacityProvider
- !Ref OnDemandCapacityProvider
DefaultCapacityProviderStrategy:
- CapacityProvider: !Ref SpotCapacityProvider
Base: 2
Weight: 4
- CapacityProvider: !Ref OnDemandCapacityProvider
Weight: 1
This configuration runs 2 tasks on on-demand capacity as a baseline and distributes additional tasks with a 4:1 ratio favoring Spot capacity, significantly reducing costs while maintaining a safety net of on-demand capacity.
Testing Your Scaling Configuration
Before trusting your scaling configuration in production, you should test it under controlled load. Use a load testing tool to simulate traffic and observe how your service responds. Here is a simple load testing script using Python and asyncio:
import asyncio
import aiohttp
import time
import statistics
async def make_request(session, url, results):
start = time.time()
try:
async with session.get(url, timeout=10) as response:
elapsed = time.time() - start
results.append({
'status': response.status,
'time': elapsed
})
except Exception as e:
results.append({
'status': 0,
'time': time.time() - start,
'error': str(e)
})
async def load_test(url, total_requests, concurrency):
results = []
connector = aiohttp.TCPConnector(limit=concurrency)
async with aiohttp.ClientSession(connector=connector) as session:
batch_size = concurrency
for i in range(0, total_requests, batch_size):
batch = [
make_request(session, url, results)
for _ in range(min(batch_size, total_requests - i))
]
await asyncio.gather(*batch)
print(f'Completed {min(i + batch_size, total_requests)}/{total_requests}')
return results
def analyze_results(results):
successful = [r for r in results if r['status'] == 200]
failed = [r for r in results if r['status'] != 200]
times = [r['time'] for r in successful]
print(f'\n--- Load Test Results ---')
print(f'Total requests: {len(results)}')
print(f'Successful: {len(successful)}')
print(f'Failed: {len(failed)}')
print(f'Success rate: {len(successful)/len(results)*100:.1f}%')
print(f'Average response time: {statistics.mean(times)*1000:.0f}ms')
print(f'P50: {statistics.median(times)*1000:.0f}ms')
print(f'P95: {statistics.quantiles(times, n=20)[18]*1000:.0f}ms')
print(f'P99: {statistics.quantiles(times, n=100)[98]*1000:.0f}ms')
if __name__ == '__main__':
url = 'https://your-alb-dns-name/api'
total_requests = 10000
concurrency = 100
results = asyncio.run(load_test(url, total_requests, concurrency))
analyze_results(results)
Run this test while monitoring your CloudWatch dashboard. You should observe the CPU utilization rising, triggering scale-out events, and new tasks being registered with the load balancer. After the test completes, watch the scale-in behavior to ensure tasks are terminated appropriately without being too aggressive.
Conclusion
Scaling a Fargate application from prototype to production requires careful planning across multiple dimensions: resource allocation, auto scaling policies, load balancer configuration, monitoring, deployment strategies, and cost optimization. By following the practices outlined in this tutorial — right-sizing tasks, using target tracking and custom metric scaling, implementing graceful shutdown and circuit breakers, distributing across availability zones, and continuously monitoring and testing your configuration — you can build a Fargate deployment that is resilient, performant, and cost-effective. Remember that scaling is not a one-time configuration but an ongoing process of measurement, adjustment, and refinement as your application and traffic patterns evolve over time.