Scaling App Runner: From Prototype to Production

Scaling App Runner: From Prototype to Production

AWS App Runner is a fully managed container application service that lets you build, deploy, and run containerized web applications and APIs without managing infrastructure. It's an excellent choice for moving quickly from prototype to production, but as your application grows, you need to understand how to scale it effectively. This tutorial walks you through everything you need to know to take an App Runner service from a simple prototype to a robust, production-ready deployment.

What Is App Runner Scaling?

App Runner automatically scales your application based on incoming traffic. When you deploy a service, App Runner provisions instances (containers) to handle requests. As traffic increases, it adds more instances; as traffic decreases, it removes them. This is conceptually similar to AWS Fargate or Elastic Beanstalk, but with far less configuration overhead.

Scaling in App Runner is governed by two main dimensions:

• Auto scaling configuration — controls how aggressively instances are added or removed based on concurrent requests per instance.
• Instance configuration — controls the CPU and memory allocated to each individual instance.

By default, App Runner uses sensible values: it scales based on a target of 100 concurrent requests per instance, with a minimum of 1 instance and a maximum of 25. For a prototype, this is fine. For production, you will almost certainly need to tune these values.

Why Scaling Matters

When you move from prototype to production, several things change. Traffic becomes unpredictable. Users expect low latency. Costs need to be controlled. Downtime is unacceptable. App Runner's default scaling configuration is optimized for getting started, not for handling production workloads.

Consider a common scenario: you launch a prototype with the default configuration. It works fine in testing with a handful of users. Then you launch to real users, traffic spikes, and App Runner scales up to the default maximum of 25 instances. If each instance can handle 100 concurrent requests, that's 2,500 concurrent requests — which may not be enough. Users start seeing timeouts and errors. Alternatively, you might have a low-traffic service where the default minimum of 1 instance means cold starts every time traffic arrives after a quiet period.

Tuning scaling configuration lets you balance cost, performance, and reliability for your specific workload.

How to Configure Scaling

You can configure App Runner scaling through the AWS Management Console, the AWS CLI, or infrastructure-as-code tools like AWS CloudFormation or Terraform. For production deployments, infrastructure-as-code is strongly recommended because it makes your configuration reproducible and version-controlled.

Creating an Auto Scaling Configuration with the AWS CLI

The first step is to create a custom auto scaling configuration. Here is an example that sets a minimum of 2 instances (to avoid cold starts), a maximum of 50 instances, and tunes the concurrency target to 80 requests per instance:

aws apprunner create-auto-scaling-configuration \
  --auto-scaling-configuration-name production-scaling-config \
  --max-concurrency 80 \
  --min-size 2 \
  --max-size 50 \
  --tags Key=Environment,Value=Production

This command returns an AutoScalingConfigurationArn, which you will need when creating or updating your App Runner service. Note that once a configuration is created, it is immutable — to change values, you create a new version and associate it with your service.

Creating a Service with a Custom Scaling Configuration

When creating a new App Runner service, you can reference your custom scaling configuration directly:

aws apprunner create-service \
  --service-name my-production-api \
  --source-configuration '{
    "ImageRepository": {
      "ImageIdentifier": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-api:latest",
      "ImageRepositoryType": "ECR",
      "ImageConfiguration": {
        "Port": "8080",
        "RuntimeEnvironmentVariables": [
          {"Name": "ENV", "Value": "production"},
          {"Name": "LOG_LEVEL", "Value": "info"}
        ]
      }
    },
    "AutoDeploymentsEnabled": true
  }' \
  --instance-configuration '{
    "Cpu": "2 vCPU",
    "Memory": "4 GB"
  }' \
  --auto-scaling-configuration-arn arn:aws:apprunner:us-east-1:123456789012:autoscalingconfiguration/production-scaling-config/1/abc123def456

Notice the instance-configuration parameter. Each instance gets 2 vCPUs and 4 GB of memory, which is a step up from the default 1 vCPU and 2 GB. The right instance size depends on your application's memory and CPU requirements — profile your application under load to determine the optimal balance.

Updating an Existing Service's Scaling Configuration

To apply a new scaling configuration to an existing service, use the update-service command:

aws apprunner update-service \
  --service-arn arn:aws:apprunner:us-east-1:123456789012:service/my-production-api/abc123 \
  --auto-scaling-configuration-arn arn:aws:apprunner:us-east-1:123456789012:autoscalingconfiguration/production-scaling-config/2/xyz789

App Runner applies the new configuration without downtime. Existing instances continue serving traffic while new instances are provisioned with the updated settings.

Defining Scaling in CloudFormation

For production environments, CloudFormation templates make your infrastructure reproducible. Here is a complete example that defines an auto scaling configuration and an App Runner service together:

AWSTemplateFormatVersion: '2010-09-09'
Description: Production App Runner service with custom scaling

Resources:
  ScalingConfig:
    Type: AWS::AppRunner::AutoScalingConfiguration
    Properties:
      AutoScalingConfigurationName: production-scaling-config
      MaxConcurrency: 80
      MinSize: 2
      MaxSize: 50
      Tags:
        - Key: Environment
          Value: Production

  AppRunnerService:
    Type: AWS::AppRunner::Service
    Properties:
      ServiceName: my-production-api
      SourceConfiguration:
        ImageRepository:
          ImageIdentifier: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/my-api:latest"
          ImageRepositoryType: ECR
          ImageConfiguration:
            Port: "8080"
            RuntimeEnvironmentVariables:
              - Name: ENV
                Value: production
              - Name: LOG_LEVEL
                Value: info
        AutoDeploymentsEnabled: true
      InstanceConfiguration:
        Cpu: "2 vCPU"
        Memory: "4 GB"
      AutoScalingConfigurationArn: !GetAtt ScalingConfig.AutoScalingConfigurationArn
      HealthCheckConfiguration:
        Protocol: HTTP
        Path: /health
        Interval: 10
        Timeout: 5
        HealthyThreshold: 1
        UnhealthyThreshold: 5

Outputs:
  ServiceUrl:
    Value: !GetAtt AppRunnerService.ServiceUrl

This template also includes a HealthCheckConfiguration, which is critical for production. App Runner uses health checks to determine whether an instance is healthy enough to receive traffic. The default health check hits the root path, but defining a dedicated /health endpoint gives you more control and avoids false positives from application logic on the root path.

Defining Scaling in Terraform

If you prefer Terraform, the equivalent configuration looks like this:

resource "aws_apprunner_auto_scaling_configuration_version" "production" {
  auto_scaling_configuration_name = "production-scaling-config"
  max_concurrency                 = 80
  min_size                        = 2
  max_size                        = 50

  tags = {
    Environment = "Production"
  }
}

resource "aws_apprunner_service" "api" {
  service_name = "my-production-api"

  source_configuration {
    image_repository {
      image_identifier      = "${data.aws_caller_identity.current.account_id}.dkr.ecr.${data.aws_region.current.name}.amazonaws.com/my-api:latest"
      image_repository_type = "ECR"
      image_configuration {
        port = "8080"
        runtime_environment_variables = {
          ENV       = "production"
          LOG_LEVEL = "info"
        }
      }
    }
    auto_deployments_enabled = true
  }

  instance_configuration {
    cpu    = "2 vCPU"
    memory = "4 GB"
  }

  auto_scaling_configuration_arn = aws_apprunner_auto_scaling_configuration_version.production.arn

  health_check_configuration {
    protocol           = "HTTP"
    path               = "/health"
    interval           = 10
    timeout            = 5
    healthy_threshold  = 1
    unhealthy_threshold = 5
  }

  tags = {
    Environment = "Production"
  }
}

Best Practices for Production Scaling

Set the Right Minimum Size

The minimum size determines how many instances are always running. A minimum of 1 is fine for cost-sensitive prototypes, but it means any traffic after a quiet period hits a cold instance. For production services with latency-sensitive workloads, set the minimum to at least 2. This provides redundancy and eliminates cold-start latency for the first request after a scale-down event. For services that must always respond instantly, consider a higher minimum based on your baseline traffic.

Set a Realistic Maximum Size

The maximum size is a safety valve. It prevents runaway costs from unexpected traffic spikes or bugs that cause retry storms. Calculate your maximum based on the peak traffic you expect, then add a buffer. For example, if you expect 5,000 concurrent requests at peak and each instance handles 80 concurrent requests, you need at least 63 instances. Set the maximum to 80 to provide headroom. Always pair this with CloudWatch alarms that alert you when you approach the maximum, so you can investigate before users are affected.

Tune the Concurrency Target Carefully

The MaxConcurrency value tells App Runner how many concurrent requests a single instance should handle before App Runner provisions another instance. Setting it too high means each instance handles too many requests, leading to high latency and potential memory exhaustion. Setting it too low means App Runner provisions instances too aggressively, increasing costs. The right value depends on your application's per-request resource usage. Load test your application to find the point where latency starts to degrade, then set the concurrency target slightly below that point.

Implement a Dedicated Health Check Endpoint

Your health check endpoint should be lightweight and fast. It should verify that the application process is alive and able to serve requests, but it should not check external dependencies like databases on every call. A health check that queries the database every 10 seconds across dozens of instances can create significant load. Instead, use a simple endpoint that returns HTTP 200, and rely on separate monitoring for dependency health.

Use Custom Domains and TLS

App Runner provides a default domain with TLS, but production services should use custom domains. You can associate a custom domain with your App Runner service, and App Runner manages the TLS certificate automatically through AWS Certificate Manager. This gives users a branded URL and ensures traffic is encrypted end to end.

aws apprunner associate-custom-domain \
  --service-arn arn:aws:apprunner:us-east-1:123456789012:service/my-production-api/abc123 \
  --domain-name api.example.com \
  --enable-www-subdomain

Monitor with CloudWatch Metrics

App Runner emits several CloudWatch metrics that are essential for production monitoring. Key metrics include NumActiveInstances, NumProvisionedInstances, RequestCount, Latency, and 4xxErrorCount / 5xxErrorCount. Create dashboards and alarms for these metrics. For example, alarm when NumActiveInstances reaches 80% of your maximum, or when p99 latency exceeds your SLO.

aws cloudwatch put-metric-alarm \
  --alarm-name "AppRunner-HighInstanceCount" \
  --alarm-description "Alert when App Runner instances approach max capacity" \
  --namespace AWS/AppRunner \
  --metric-name NumActiveInstances \
  --dimensions Name=ServiceName,Value=my-production-api \
  --statistic Maximum \
  --period 60 \
  --evaluation-periods 2 \
  --threshold 40 \
  --comparison-greater-than-threshold \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:alerts

Enable Deployment Safety with Auto Rollback

App Runner supports automatic deployments from ECR or source code repositories. In production, enable auto rollback so that if a new deployment fails health checks, App Runner automatically reverts to the previous healthy version. This is configured at the service level and prevents bad deployments from taking down your service.

Consider VPC Ingress for Internal Services

By default, App Runner services are publicly accessible. For internal APIs that should not be exposed to the internet, configure VPC ingress. This allows traffic to your App Runner service only from within your VPC or through private network paths. This is particularly important for microservices that communicate with each other but should never be reachable from the public internet.

Optimize Container Image Size

Smaller container images start faster, which directly impacts scaling responsiveness. When App Runner scales up, it pulls your container image and starts a new instance. A 1 GB image takes significantly longer to pull than a 100 MB image. Use multi-stage builds, minimal base images like Amazon Linux 2 or Alpine, and exclude unnecessary files to keep your image as small as possible.

# Multi-stage Dockerfile example
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .
RUN npm run build

FROM node:18-alpine AS runtime
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package.json ./
EXPOSE 8080
CMD ["node", "dist/index.js"]

Handle Graceful Shutdown

When App Runner scales down, it sends a SIGTERM signal to the container and waits a short period before sending SIGKILL. Your application should listen for SIGTERM, stop accepting new requests, finish in-flight requests, and then exit cleanly. This prevents users from experiencing errors during scale-down events.

// Node.js graceful shutdown example
process.on('SIGTERM', () => {
  console.log('SIGTERM received, shutting down gracefully');
  server.close(() => {
    console.log('HTTP server closed');
    process.exit(0);
  });

  // Force exit after 30 seconds if server hasn't closed
  setTimeout(() => {
    console.error('Forcing shutdown after timeout');
    process.exit(1);
  }, 30000);
});

Load Test Before Launching

Never guess your scaling configuration. Before launching to production, load test your application using a tool like Artillery, k6, or AWS Distributed Load Testing. Start with a small number of users and gradually increase. Monitor CPU utilization, memory usage, response latency, and error rates. Use the results to determine the optimal MaxConcurrency, instance size, and maximum instance count. Load testing also validates that your application handles concurrent requests correctly and that your database or downstream services can keep up.

Conclusion

Scaling an App Runner service from prototype to production is about more than just increasing instance counts. It requires thoughtful tuning of concurrency targets, instance sizes, and minimum and maximum bounds based on real load testing data. It requires infrastructure-as-code for reproducibility, proper health checks for reliability, monitoring for observability, and graceful shutdown handling for a smooth user experience during scaling events. By following the practices and patterns in this tutorial, you can confidently take an App Runner service from a quick prototype to a production deployment that is cost-efficient, responsive under load, and resilient to traffic spikes. Start with the defaults, measure your application's behavior, and iterate on your configuration as your traffic patterns become clear.