← Back to DevBytes
Cloud August 04, 2026 11 min read

Scaling CloudRun: From Prototype to Production

Introduction to Scaling Cloud Run

Google Cloud Run has become one of the most popular serverless container platforms for deploying stateless applications. Its simplicity makes it incredibly easy to take a prototype from local development to a deployed service in minutes. However, moving from a prototype that handles a handful of requests to a production-grade service that serves thousands of concurrent users requires a deeper understanding of how Cloud Run scales, how to configure it properly, and how to avoid common pitfalls.

In this tutorial, we will walk through the entire journey of scaling a Cloud Run service. We will cover the fundamentals of how Cloud Run autoscaling works, how to configure concurrency and instance counts, how to handle cold starts, how to optimize container startup time, and how to implement best practices for observability, security, and cost management. By the end, you will have a clear blueprint for taking your Cloud Run service to production.

What Is Cloud Run Scaling?

Cloud Run is a managed compute platform that automatically scales containerized applications up and down based on incoming request traffic. Each Cloud Run service runs as a set of stateless container instances. When traffic increases, Cloud Run spins up additional instances. When traffic decreases, it removes instances down to zero if configured to do so.

The key scaling concepts in Cloud Run include:

Understanding how these parameters interact is essential for building a production-ready service. The right configuration depends on your application's latency requirements, traffic patterns, and budget constraints.

Why Scaling Configuration Matters

When you first deploy a Cloud Run service, the defaults work fine for prototyping. But in production, misconfigured scaling parameters can lead to several serious problems. If concurrency is set too high for your application, a single instance may become overwhelmed, leading to increased latency and timeouts. If max instances is not set, a traffic spike could spin up hundreds of instances, overwhelming downstream databases and driving up costs dramatically.

Cold starts are another critical concern. If your service scales to zero during quiet periods, the first request after a period of inactivity will experience a cold start delay. For user-facing APIs, this can mean seconds of latency, which is unacceptable. For background job processors, it may be perfectly fine.

Proper scaling configuration also affects reliability. Without a max instances limit, a sudden burst of traffic can cascade failures into your database connection pool or external APIs. With thoughtful configuration, Cloud Run can absorb traffic spikes gracefully while protecting your downstream dependencies.

Configuring Concurrency

Concurrency is one of the most important parameters to tune. The default value of 80 works well for many applications, but the optimal value depends on how much CPU and memory each request consumes. CPU-bound applications should use lower concurrency, while I/O-bound applications that spend most of their time waiting on external services can handle much higher concurrency.

You can set concurrency when deploying a service:

gcloud run deploy my-service \
  --image gcr.io/my-project/my-service:latest \
  --region us-central1 \
  --concurrency 100 \
  --platform managed

You can also update concurrency on an existing service without redeploying the image:

gcloud run services update my-service \
  --concurrency 200 \
  --region us-central1

To determine the right concurrency for your application, load test with different values and monitor CPU utilization, memory usage, and request latency. A good rule of thumb is to start with the default of 80, then increase it for I/O-bound services while watching for latency degradation. If your application uses a thread pool or connection pool internally, make sure the pool size is aligned with your concurrency setting.

Setting Min and Max Instances

Min instances keeps a baseline number of instances always running. This is the primary tool for eliminating cold starts. The tradeoff is cost: you pay for these instances even when there is no traffic. For services with strict latency requirements, setting min instances to 1 or 2 is often worth the cost.

Max instances caps the number of instances Cloud Run will create. This is a critical safety mechanism for production. Without it, a traffic spike or a bug causing retry storms can result in unexpectedly large bills and cascading failures in downstream systems.

gcloud run deploy my-service \
  --image gcr.io/my-project/my-service:latest \
  --region us-central1 \
  --min-instances 2 \
  --max-instances 50 \
  --concurrency 100

When choosing max instances, consider the capacity of your downstream dependencies. If each instance opens 10 database connections and your database can handle 500 connections, your max instances should not exceed 50. Always calculate the maximum load your backend systems can tolerate and set max instances accordingly.

You can also use the Google Cloud Console or a YAML service definition to configure these settings. Here is an example of a service YAML that can be applied with gcloud run services replace:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: my-service
  spec:
    template:
      metadata:
        annotations:
          autoscaling.knative.dev/minScale: "2"
          autoscaling.knative.dev/maxScale: "50"
          run.googleapis.com/execution-environment: gen2
      spec:
        containerConcurrency: 100
        containers:
        - image: gcr.io/my-project/my-service:latest
          resources:
            limits:
              cpu: "2"
              memory: "1Gi"

Handling Cold Starts

Cold starts occur when Cloud Run needs to create a new container instance to handle a request. The cold start time includes the time to pull the container image, start the container, and run your application's initialization code. For production services, cold starts can cause noticeable latency spikes that degrade user experience.

There are several strategies to reduce cold start impact:

To enable CPU always allocated, use the following flag:

gcloud run services update my-service \
  --cpu always \
  --region us-central1

Here is an example of a multi-stage Dockerfile that produces a small image for faster cold starts:

# Build stage
FROM node:18-slim AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .

# Runtime stage
FROM node:18-slim
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app .
EXPOSE 8080
CMD ["node", "server.js"]

Optimizing CPU and Memory Allocation

Cloud Run allows you to configure the CPU and memory allocated to each container instance. The right allocation depends on your application's workload. CPU-bound services like image processing or data transformation need more CPU, while memory-intensive services like in-memory caching need more RAM.

Cloud Run supports a ratio of CPU to memory. You can specify CPU in increments of 1 (for the first vCPU) and then in finer increments, and memory in increments of 128MiB. Here is how to set resources:

gcloud run deploy my-service \
  --image gcr.io/my-project/my-service:latest \
  --cpu 2 \
  --memory 2Gi \
  --region us-central1

When CPU is set to always allocated, your application can perform background work between requests. This is useful for maintaining connection pools, warming caches, and running health checks. However, it also means you are billed for CPU usage even when no requests are being processed. For services with min instances set to zero, CPU is only allocated during request processing by default.

If your application is I/O-bound and handles high concurrency, consider using a single vCPU with higher concurrency rather than multiple vCPUs with lower concurrency. This can be more cost-effective since you are paying for fewer CPU resources while still handling the same total request volume.

Implementing Graceful Shutdown

When Cloud Run scales down, it sends a SIGTERM signal to your container before terminating it. Your application should listen for this signal and shut down gracefully by finishing in-flight requests, closing database connections, and releasing resources. Without graceful shutdown, in-flight requests may fail, causing errors for users.

Here is an example of graceful shutdown in a Node.js application:

const http = require('http');
const server = http.createServer(handler);

server.listen(8080, () => {
  console.log('Server listening on port 8080');
});

let isShuttingDown = false;

function handler(req, res) {
  if (isShuttingDown) {
    res.writeHead(503);
    res.end('Server is shutting down');
    return;
  }
  res.writeHead(200);
  res.end('OK');
}

process.on('SIGTERM', () => {
  console.log('SIGTERM received, shutting down gracefully');
  isShuttingDown = true;
  server.close(() => {
    console.log('All connections closed');
    process.exit(0);
  });
  
  // Force exit after 10 seconds if connections don't close
  setTimeout(() => {
    process.exit(1);
  }, 10000);
});

And here is a similar pattern in Python using Flask:

from flask import Flask
import signal
import sys

app = Flask(__name__)
is_shutting_down = False

@app.before_request
def check_shutdown():
    if is_shutting_down:
        return "Server is shutting down", 503

@app.route("/")
def hello():
    return "OK"

def handle_sigterm(signum, frame):
    global is_shutting_down
    is_shutting_down = True
    print("SIGTERM received, shutting down gracefully")
    # Give in-flight requests time to complete
    import threading
    def shutdown():
        import time
        time.sleep(5)
        sys.exit(0)
    threading.Thread(target=shutdown).start()

signal.signal(signal.SIGTERM, handle_sigterm)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Monitoring and Observability

Production services require robust observability. Cloud Run integrates with Google Cloud's operations suite, including Cloud Monitoring, Cloud Logging, and Cloud Trace. You should instrument your application with structured logging, custom metrics, and distributed tracing to gain visibility into performance and scaling behavior.

Structured logging makes it easier to query and filter logs. Here is an example of structured logging in Node.js that is compatible with Cloud Logging:

function logJson(severity, message, extra = {}) {
  const entry = {
    severity,
    message,
    ...extra,
    timestamp: new Date().toISOString()
  };
  console.log(JSON.stringify(entry));
}

// Usage
logJson('INFO', 'Request processed', {
  requestId: req.id,
  durationMs: 42,
  endpoint: '/api/users'
});

You should also set up alerting policies in Cloud Monitoring to notify you when key metrics cross thresholds. For example, you can alert when the number of instances approaches your max instances limit, when latency exceeds your SLO, or when error rates spike. Here is an example of creating an alert policy using the gcloud CLI:

gcloud alpha monitoring policies create --policy-from-file=alert-policy.yaml

With an alert policy YAML like this:

displayName: High Latency Alert
conditions:
  - displayName: p99 latency above 500ms
    conditionThreshold:
      filter: |
        resource.type="cloud_run_revision" AND
        resource.label.service_name="my-service" AND
        metric.type="run.googleapis.com/request_latencies"
      comparison: COMPARISON_GT
      thresholdValue: 500
      duration: 300s
      aggregations:
        - alignmentPeriod: 60s
          perSeriesAligner: ALIGN_PERCENTILE_99
combiner: OR
notificationChannels:
  - projects/my-project/notificationChannels/123456

Best Practices for Production

Beyond the core scaling parameters, several best practices will help ensure your Cloud Run service is production-ready:

Here is an example of deploying a service with a VPC connector and Secret Manager integration:

gcloud run deploy my-service \
  --image gcr.io/my-project/my-service:latest \
  --region us-central1 \
  --vpc-connector my-connector \
  --vpc-egress private-ranges-only \
  --set-secrets="DB_PASSWORD=my-secret:latest" \
  --min-instances 2 \
  --max-instances 30 \
  --concurrency 80 \
  --cpu 1 \
  --memory 512Mi \
  --timeout 60 \
  --no-allow-unauthenticated

Load Testing Your Configuration

Before going to production, you should load test your Cloud Run service to validate your scaling configuration. Tools like k6, Artillery, or Apache JMeter can generate controlled traffic to see how your service behaves under load. Pay attention to how quickly instances spin up, whether latency stays within your SLO, and whether your downstream systems can handle the load.

Here is a simple k6 load test script:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 50 },
    { duration: '1m', target: 100 },
    { duration: '30s', target: 200 },
    { duration: '1m', target: 200 },
    { duration: '30s', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(99)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://my-service-abc123-uc.a.run.app/api/health');
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
  sleep(0.1);
}

Run the test and monitor the Cloud Run metrics in the console. Look at the instance count over time, the request latency distribution, and the error rate. Adjust your concurrency, min instances, and max instances based on the results. Iterate until your service meets your performance and reliability targets.

Conclusion

Scaling a Cloud Run service from prototype to production is about more than just deploying a container. It requires thoughtful configuration of concurrency, instance limits, CPU and memory allocation, and graceful shutdown handling. It requires optimizing container images for fast cold starts, implementing robust observability, and load testing under realistic conditions. By following the practices outlined in this tutorial, you can confidently take your Cloud Run service to production, knowing it will scale efficiently, handle traffic spikes gracefully, protect your downstream dependencies, and keep costs under control. The journey from prototype to production is iterative, so start with sensible defaults, measure continuously, and refine your configuration as you learn how your application behaves under real-world traffic.

— Ad —

Google AdSense will appear here after approval

← Back to all articles