Scaling ElastiCache: From Prototype to Production

Scaling ElastiCache: From Prototype to Production

Amazon ElastiCache is a fully managed in-memory data store service that supports Redis and Memcached. It is the go-to solution for developers who need sub-millisecond latency for caching, session storage, leaderboards, real-time analytics, and more. While spinning up a single-node ElastiCache cluster for a prototype is trivial, scaling that same setup to handle production traffic is an entirely different challenge. This tutorial walks you through the journey from a basic prototype to a resilient, high-throughput production cache layer.

Why Scaling ElastiCache Matters

In the prototype phase, a single cache node is usually sufficient. You have low traffic, no strict availability requirements, and data loss is tolerable. As you move toward production, several concerns emerge:

• Throughput: A single node has finite CPU and network bandwidth. Production workloads often require millions of operations per second.
• Availability: A single node is a single point of failure. You need replication and automatic failover.
• Memory limits: Individual nodes cap out at a few hundred gigabytes. Sharding lets you exceed that ceiling.
• Latency consistency: Under load, a single node can become a bottleneck, causing tail latency spikes.
• Cost efficiency: Over-provisioning large nodes is expensive. Right-sizing and horizontal scaling are more economical.

Understanding these concerns shapes every architectural decision you make when promoting your cache to production.

Understanding ElastiCache Architecture

Before scaling, you need to understand the two engines and their scaling characteristics. Memcached is a simple key-value store with no persistence, no replication, and no built-in sharding logic — clients handle distribution. Redis (specifically Redis OSS and Redis Cluster mode in ElastiCache) supports replication, persistence, pub/sub, and server-side sharding.

For most modern production workloads, Redis with Cluster Mode Enabled (CME) is the recommended choice. It provides automatic sharding across multiple shards, each with its own primary and read replicas, plus automatic failover. This tutorial focuses primarily on Redis with Cluster Mode Enabled.

Key Concepts

• Node: The smallest unit, a single EC2 instance running Redis.
• Shard (Node Group): A primary node plus zero or more read replicas. In cluster mode, data is partitioned across shards.
• Cluster: A collection of shards. In cluster mode, the cluster manages 16,384 hash slots distributed across shards.
• Replication Group: The ElastiCache abstraction for a Redis cluster with replication.
• Endpoint types: Primary endpoint, reader endpoint, node endpoints, and configuration endpoint (cluster mode).

Starting Point: The Prototype

Let's begin with a typical prototype setup: a single Redis node with no replication. Here is how you might create it using the AWS CLI:

aws elasticache create-cache-cluster \
  --cache-cluster-id prototype-redis \
  --engine redis \
  --cache-node-type cache.t3.micro \
  --num-cache-nodes 1 \
  --engine-version 7.0

And here is a simple Python client using redis-py to interact with it:

import redis

# Single node endpoint
r = redis.Redis(
    host="prototype-redis.xxxxxx.0001.use1.cache.amazonaws.com",
    port=6379,
    decode_responses=True
)

# Basic cache-aside pattern
def get_user(user_id):
    cache_key = f"user:{user_id}"
    cached = r.get(cache_key)
    if cached:
        return cached

    # Cache miss — fetch from database
    user = db.fetch_user(user_id)
    r.setex(cache_key, 3600, user)  # TTL of 1 hour
    return user

This works fine for development, but it has no failover, no scalability beyond one node, and no read scaling. Let's fix that.

Step 1: Enable Replication and Automatic Failover

The first production upgrade is moving from a standalone cache cluster to a replication group. A replication group gives you a primary node for writes and one or more read replicas. If the primary fails, ElastiCache automatically promotes a replica.

aws elasticache create-replication-group \
  --replication-group-id prod-redis-rg \
  --replication-group-description "Production Redis with replication" \
  --engine redis \
  --cache-node-type cache.r6g.large \
  --num-cache-clusters 2 \
  --automatic-failover-enabled \
  --multi-az-enabled \
  --engine-version 7.0 \
  --parameter-group-name default.redis7 \
  --notification-topic-arn arn:aws:sns:us-east-1:123456789012:cache-alerts

Key parameters to note:

• --num-cache-clusters 2 creates one primary and one replica.
• --automatic-failover-enabled allows ElastiCache to promote a replica if the primary dies.
• --multi-az-enabled places the primary and replica in different Availability Zones.
• cache.r6g.large uses memory-optimized instances, which are better suited for Redis than the burstable t3 family.

Your client code needs minimal changes. Use the primary endpoint for writes and the reader endpoint for reads:

import redis

PRIMARY = "prod-redis-rg.xxxxxx.0001.use1.cache.amazonaws.com"
READER = "prod-redis-rg-ro.xxxxxx.0001.use1.cache.amazonaws.com"

write_client = redis.Redis(host=PRIMARY, port=6379, decode_responses=True)
read_client = redis.Redis(host=READER, port=6379, decode_responses=True)

def get_user(user_id):
    cache_key = f"user:{user_id}"
    cached = read_client.get(cache_key)
    if cached:
        return cached

    user = db.fetch_user(user_id)
    write_client.setex(cache_key, 3600, user)
    return user

Step 2: Scale Horizontally with Cluster Mode

Replication handles availability and read scaling, but a single shard still limits your total memory and write throughput. To scale beyond one shard, enable Cluster Mode. With cluster mode, ElastiCache partitions your keyspace across multiple shards using Redis hash slots.

aws elasticache create-replication-group \
  --replication-group-id prod-redis-cluster \
  --replication-group-description "Production Redis Cluster Mode Enabled" \
  --engine redis \
  --cache-node-type cache.r6g.large \
  --num-node-groups 3 \
  --replicas-per-node-group 1 \
  --automatic-failover-enabled \
  --multi-az-enabled \
  --cluster-mode enabled \
  --engine-version 7.0

This creates a cluster with 3 shards, each having 1 primary and 1 replica (6 nodes total). The cluster exposes a single configuration endpoint that clients use to discover the topology.

Your client code must now use a cluster-aware client. With redis-py, use RedisCluster:

from redis.cluster import RedisCluster

CONFIG_ENDPOINT = "prod-redis-cluster.xxxxxx.clustercfg.use1.cache.amazonaws.com"

rc = RedisCluster(
    host=CONFIG_ENDPOINT,
    port=6379,
    decode_responses=True,
    ssl=True,
    socket_timeout=2,
    socket_connect_timeout=2,
    retry_on_timeout=True
)

def get_user(user_id):
    cache_key = f"user:{user_id}"
    cached = rc.get(cache_key)
    if cached:
        return cached

    user = db.fetch_user(user_id)
    rc.setex(cache_key, 3600, user)
    return user

The cluster client automatically routes each command to the correct shard based on the key's hash slot. You no longer manage primary and reader endpoints manually — the client handles it.

Important: Multi-Key Operations

In cluster mode, multi-key operations like MGET, MSET, and Lua scripts that touch multiple keys only work if all keys map to the same hash slot. You can enforce this using hash tags — wrapping a portion of the key in curly braces:

# These keys are guaranteed to land on the same shard
# because the hash tag {user:1001} determines the slot
rc.set("user:1001:profile", profile_data)
rc.set("user:1001:settings", settings_data)
rc.set("user:1001:activity", activity_data)

# This MGET works because all keys share the same hash tag
rc.mget(["user:1001:profile", "user:1001:settings", "user:1001:activity"])

Without hash tags, a multi-key operation spanning shards will raise a CROSSSLOT error. Design your key schema with this in mind from the start.

Step 3: Online Resharding and Scaling

One of the most powerful features of ElastiCache Cluster Mode is the ability to add or remove shards with zero downtime. This is called online resharding. ElastiCache migrates hash slots in the background while your application continues to serve traffic.

To add a shard to an existing cluster:

aws elasticache increase-replica-count \
  --replication-group-id prod-redis-cluster \
  --apply-immediately

To add a new node group (shard), use the modify-replication-group-shard-configuration API:

aws elasticache modify-replication-group-shard-configuration \
  --replication-group-id prod-redis-cluster \
  --node-group-count 4 \
  --apply-immediately

This scales the cluster from 3 shards to 4. ElastiCache will:

• Provision the new shard with a primary and replica.
• Reassign hash slots from existing shards to the new one.
• Migrate data in small batches to avoid overwhelming the cluster.
• Update the cluster topology so clients redirect traffic.

During resharding, monitor the IsTruncated and migration metrics in CloudWatch. The process can take anywhere from minutes to hours depending on data size and load.

Step 4: Vertical Scaling (Changing Node Type)

Sometimes you need more memory or CPU per node rather than more shards. ElastiCache supports vertical scaling by changing the node type. For Redis with cluster mode, this can be done with minimal downtime:

aws elasticache modify-replication-group \
  --replication-group-id prod-redis-cluster \
  --cache-node-type cache.r6g.2xlarge \
  --apply-immediately

ElastiCache will perform a rolling upgrade: it takes one node offline at a time, replaces it with the new instance type, waits for replication to sync, then moves to the next node. Because replicas are promoted only after sync, data loss is minimized.

For planned vertical scaling, prefer using --apply-immediately false and scheduling the change during a maintenance window to control when the brief interruptions occur.

Step 5: Security Hardening for Production

Production caches must be secured. ElastiCache provides several layers of protection that you should configure before going live.

Encryption in Transit

aws elasticache modify-replication-group \
  --replication-group-id prod-redis-cluster \
  --transit-encryption-enabled \
  --apply-immediately

This enables TLS for all client connections. Update your client to use SSL:

rc = RedisCluster(
    host=CONFIG_ENDPOINT,
    port=6379,
    ssl=True,
    ssl_cert_reqs="required",
    decode_responses=True
)

Encryption at Rest and Authentication

aws elasticache modify-replication-group \
  --replication-group-id prod-redis-cluster \
  --at-rest-encryption-enabled \
  --auth-token "YourStrongAuthTokenHere!" \
  --transit-encryption-enabled \
  --apply-immediately

Note that --auth-token (Redis AUTH) requires transit encryption to be enabled. Your client must then pass the password:

rc = RedisCluster(
    host=CONFIG_ENDPOINT,
    port=6379,
    ssl=True,
    password="YourStrongAuthTokenHere!",
    decode_responses=True
)

Network Isolation

Always deploy ElastiCache in a private subnet with no internet gateway route. Use security groups to restrict access to only your application servers:

aws ec2 authorize-security-group-ingress \
  --group-id sg-elasticache \
  --protocol tcp \
  --port 6379 \
  --source-group sg-application

Step 6: Monitoring and Observability

A production cache without monitoring is a liability. ElastiCache integrates with CloudWatch and provides dozens of metrics. The most critical ones to alert on are:

• CPUUtilization — high CPU means the node is saturated. For Redis, this is single-threaded per core, so watch it closely.
• Evictions — non-zero evictions mean your cache is full and keys are being removed. Either increase memory or review TTLs.
• CacheHits and CacheMisses — calculate your hit ratio. Below 80-90% suggests your cache strategy needs review.
• ReplicationLag — lag between primary and replicas. Sustained lag indicates network or load issues.
• SwapUsage — any swap usage is bad for an in-memory store and indicates memory pressure.
• CurrentConnections — track connection pool health and detect leaks.

Here is a CloudWatch alarm for high eviction rates:

aws cloudwatch put-metric-alarm \
  --alarm-name "ElastiCache-HighEvictions" \
  --alarm-description "Alert when evictions exceed threshold" \
  --namespace "AWS/ElastiCache" \
  --metric-name "Evictions" \
  --dimensions Name=CacheClusterId,Value=prod-redis-cluster \
  --statistic "Sum" \
  --period 300 \
  --threshold 1000 \
  --comparison-operator "GreaterThanThreshold" \
  --evaluation-periods 2 \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:cache-alerts

You can also enable slow logs to identify problematic queries:

aws elasticache modify-cache-parameter-group \
  --cache-parameter-group-name prod-redis-params \
  --parameter-name-values ParameterName=slowlog-log-slower-than,ParameterValue=10000 \
  --parameter-name-values ParameterName=slowlog-max-len,ParameterValue=1024

This logs any command taking longer than 10 milliseconds.

Best Practices for Production ElastiCache

Design for Cache Misses

Your application must gracefully handle cache misses and failures. Never let a cache outage take down your service. Use circuit breakers and fallbacks:

import redis
from redis.cluster import RedisCluster
import time

class CacheService:
    def __init__(self, config_endpoint):
        self.client = RedisCluster(
            host=config_endpoint,
            port=6379,
            ssl=True,
            socket_timeout=1,
            socket_connect_timeout=1,
            decode_responses=True
        )
        self._circuit_open = False
        self._circuit_opened_at = 0

    def _check_circuit(self):
        if self._circuit_open:
            if time.time() - self._circuit_opened_at > 30:
                self._circuit_open = False  # Half-open: try again
            else:
                return False
        return True

    def _trip_circuit(self):
        self._circuit_open = True
        self._circuit_opened_at = time.time()

    def get(self, key):
        if not self._check_circuit():
            return None
        try:
            return self.client.get(key)
        except (redis.ConnectionError, redis.TimeoutError):
            self._trip_circuit()
            return None

    def set(self, key, value, ttl=3600):
        if not self._check_circuit():
            return False
        try:
            return self.client.setex(key, ttl, value)
        except (redis.ConnectionError, redis.TimeoutError):
            self._trip_circuit()
            return False

Always Set TTLs

Every key should have a TTL. Without TTLs, your cache will grow unbounded and eventually evict important keys or run out of memory. Choose TTLs based on data freshness requirements:

# Session data — 30 minutes
rc.setex("session:abc123", 1800, session_data)

# User profile — 1 hour
rc.setex("user:1001:profile", 3600, profile)

# Product catalog — 24 hours
rc.setex("product:catalog:v2", 86400, catalog_json)

# Rate limiting counter — 60 seconds
rc.setex("ratelimit:user:1001", 60, "1")

Use Connection Pooling

Creating a new connection for every request is expensive. Use connection pooling and reuse clients across your application lifecycle:

from redis.cluster import RedisCluster
from redis.connection import ConnectionPool

# Create the pool once at application startup
pool = ConnectionPool(
    host=CONFIG_ENDPOINT,
    port=6379,
    ssl=True,
    max_connections=50,
    socket_timeout=2,
    socket_connect_timeout=2,
    retry_on_timeout=True
)

rc = RedisCluster(connection_pool=pool, decode_responses=True)

Choose the Right Node Type

Redis is largely single-threaded for command execution. A node with more CPU cores does not proportionally increase throughput. Memory capacity and network bandwidth are usually the deciding factors. The r6g family is optimized for Redis workloads. Start with cache.r6g.large and scale based on observed metrics.

Plan for Failover

Automatic failover typically takes 30-60 seconds. Your application must handle connection drops during this window. Configure your client with sensible retry logic:

from redis.cluster import RedisCluster
from redis.backoff import ExponentialBackoff
from redis.retry import Retry

retry = Retry(ExponentialBackoff(cap=10, base=1), 3)

rc = RedisCluster(
    host=CONFIG_ENDPOINT,
    port=6379,
    ssl=True,
    retry=retry,
    retry_on_error=[redis.ConnectionError, redis.TimeoutError],
    decode_responses=True
)

Use Read Replicas for Read-Heavy Workloads

If your workload is read-heavy (most cache workloads are), add more replicas per shard. Each replica can serve reads, linearly scaling read throughput:

aws elasticache modify-replication-group-shard-configuration \
  --replication-group-id prod-redis-cluster \
  --node-group-count 3 \
  --replicas-per-node-group 3 \
  --apply-immediately

This gives you 3 shards with 1 primary and 3 replicas each — 12 nodes total, with 9 replicas available for reads.

Back Up Your Data

Even though cache data is theoretically disposable, backing it up can save you from a thundering herd problem after a full cache flush. Enable automatic snapshots:

aws elasticache modify-replication-group \
  --replication-group-id prod-redis-cluster \
  --snapshot-window "03:00-05:00" \
  --snapshot-retention-limit 7 \
  --automatic-failover-enabled \
  --apply-immediately

This takes daily snapshots during a low-traffic window and retains them for 7 days.

Cost Optimization Considerations

Production ElastiCache can become expensive. Here are strategies to keep costs in check:

• Right-size nodes: Monitor memory and CPU utilization. If nodes are consistently below 60% utilization, consider downsizing.
• Use reserved nodes: For steady-state production workloads, purchase Reserved Instances for 30-50% savings over on-demand pricing.
• Review replica count: More replicas improve read throughput and availability but multiply costs. Start with 1 replica per shard and add only when needed.
• Optimize data structures: Use Redis hashes instead of many small string keys. A hash with 100 fields uses far less memory than 100 separate keys.
• Enable compression: Compress large values before storing them in Redis to reduce memory footprint.

Here is an example of using a Redis hash to store user data efficiently:

# Inefficient: many separate keys
rc.set("user:1001:name", "Alice")
rc.set("user:1001:email", "alice@example.com")
rc.set("user:1001:age", "30")

# Efficient: single hash with multiple fields
rc.hset("user:1001", mapping={
    "name": "Alice",
    "email": "alice@example.com",
    "age": "30"
})
rc.expire("user:1001", 3600)

# Retrieve individual fields
name = rc.hget("user:1001", "name")

# Retrieve all fields
user_data = rc.hgetall("user:1001")

Migration Strategy: Zero-Downtime Cutover

When moving from your prototype to a production cluster, you need a migration strategy that avoids downtime. The recommended approach is a dual-write and gradual cutover:

class DualWriteCache:
    def __init__(self, old_client, new_client):
        self.old = old_client
        self.new = new_client
        self.read_from_new = False  # Toggle via feature flag

    def get(self, key):
        if self.read_from_new:
            value = self.new.get(key)
            if value is not None:
                return value
            # Fallback to old cache on miss during migration
            return self.old.get(key)
        return self.old.get(key)

    def set(self, key, value, ttl=3600):
        # Write to both caches during migration
        self.old.setex(key, ttl, value)
        try:
            self.new.setex(key, ttl, value)
        except Exception:
            pass  # Don't fail writes if new cache is unavailable

    def cutover(self):
        """Call this after verifying the new cache is warm and healthy."""
        self.read_from_new = True

The migration phases are:

• Phase 1: Dual-write to both caches, read from old.
• Phase 2: Dual-write, read from new with fallback to old.
• Phase 3: Dual-write, read only from new.
• Phase 4: Write only to new, decommission old.

Each phase should run for a observation period (hours to days) before advancing. This gives you time to catch issues without impacting users.

Conclusion

Scaling ElastiCache from a prototype to a production-grade system involves far more than just upgrading the node type. It requires thoughtful architecture decisions around replication, sharding, security, observability, and application resilience. By starting with a replication group for high availability, enabling cluster mode for horizontal scalability, hardening security with encryption and network isolation, implementing robust monitoring, and designing your application to gracefully handle cache failures, you build a cache layer that can handle production traffic with confidence. The key principle throughout is to treat your cache as a critical infrastructure component — not a disposable afterthought — and to design both the infrastructure and the application code with failure scenarios in mind. With the patterns and practices outlined in this tutorial, you are well-equipped to operate ElastiCache at production scale while maintaining performance, availability, and cost efficiency.