ElastiCache Redis Cluster Mode and Replication

Amazon ElastiCache for Redis offers two deployment modes: cluster mode disabled (single shard) and cluster mode enabled (multi-shard). Cluster mode enabled provides horizontal scalability by automatically partitioning your data across up to 500 shards (for Redis 7.x, earlier versions up to 500 shards as well). Each shard is a Redis node group consisting of a primary node and up to five read replicas. Replication provides high availability: if the primary node fails, a replica is automatically promoted to primary, ensuring minimal downtime.

Redis cluster uses a hash slot mechanism to distribute data. The entire keyspace is divided into 16,384 hash slots. Each node in the cluster is responsible for a subset of these slots. When a key is written, the client computes the CRC16 hash of the key modulo 16,384 to determine which slot the key belongs to. The client then routes the request to the node responsible for that slot. This sharding logic is implemented on the client side using the cluster's slot-to-node mapping.

Redis replication is asynchronous by default. The primary node sends a stream of commands to replicas. Replicas acknowledge receipt but not application. This means there is a potential for data loss if the primary fails before the replica receives the latest writes. ElastiCache mitigates this with Multi-AZ placement and automatic failover. For cluster mode disabled, you must enable Multi-AZ to get automatic failover. For cluster mode enabled, Multi-AZ is always enabled.

You can create a cluster with cluster mode enabled using the AWS CLI:

aws elasticache create-replication-group \
    --replication-group-id my-redis-cluster \
    --replication-group-description "Redis cluster with shards" \
    --engine redis \
    --engine-version 7.0 \
    --cache-node-type cache.r5.large \
    --num-node-groups 3 \
    --replicas-per-node-group 2 \
    --multi-az-enabled \
    --automatic-failover-enabled

To verify cluster mode status:

aws elasticache describe-replication-groups --replication-group-id my-redis-cluster

Look for the ClusterEnabled field: true means cluster mode enabled.

A large e-commerce platform uses ElastiCache Redis cluster mode to store user session data. With millions of concurrent users, a single-shard Redis would be insufficient in terms of memory and throughput. They deploy a cluster with 10 shards, each with 2 replicas, using cache.r5.large nodes. The cluster is spread across 3 AZs for high availability. The application uses a Redis cluster-aware client (e.g., Lettuce or Jedis) that handles slot mapping and redirections. During peak shopping seasons, the cluster handles over 1 million writes per second with sub-millisecond latency. If a primary node fails, automatic failover promotes a replica within seconds, and the session data remains available. The ops team monitors ReplicationLag and CacheMisses to detect issues. Misconfiguration: Initially, they used cluster mode disabled with a single shard, causing memory exhaustion and high latency. Migrating to cluster mode resolved the scalability bottleneck.

A gaming company uses ElastiCache Redis to maintain real-time leaderboards for millions of players. They use Redis sorted sets to store player scores. With cluster mode enabled, they shard data by player region to keep related data together. They use hash tags (e.g., {region}:player:123) to ensure all keys for a region are in the same shard, allowing atomic operations on the sorted set. The cluster has 20 shards across 3 AZs. They configure read replicas to handle read-heavy leaderboard queries. During a game launch, write traffic spikes; the cluster scales by adding shards (though this requires migration). A common mistake is not using hash tags, causing multi-key operations to fail across shards. They also learned to set appropriate timeouts to avoid connection buildup.

A financial services company uses ElastiCache as a cache for microservices that access a relational database. They use cluster mode enabled to provide a large cache pool (e.g., 500 GB total). Each microservice connects to the same cluster but uses key prefixes to avoid collisions. They use ElastiCache's encryption at rest and in transit to meet compliance requirements. They also enable automatic backups with 7-day retention. A critical performance consideration is that adding replicas increases read throughput but does not increase write throughput (writes still go to primary). They monitor network bandwidth and CPU to ensure nodes are not overloaded. Misconfiguration: They initially placed all replicas in the same AZ, causing data loss risk during AZ failure. They later spread replicas across AZs.