Hot Keys: Detecting and Solving Cache Hotspot Problems

Hot Keys: Detecting and Solving Cache Hotspot Problems

In any caching system, a small percentage of keys typically receive a disproportionate share of traffic. When this imbalance becomes extreme — a single key receiving thousands or millions of requests per second — that key becomes a "hot key." Hot keys are a critical challenge in distributed caching because they concentrate load on a single cache node, creating a bottleneck that undermines the entire purpose of distributing your cache.

What Makes a Key "Hot"?

A hot key is any cache key that receives significantly more traffic than the average key. In a Redis Cluster with millions of keys, if one key handles 50% of all requests, that key — and the node storing it — becomes the system's bottleneck.

Common sources of hot keys:

• Viral content: A trending tweet, a viral product listing, or breaking news article that millions of users access simultaneously.
• Celebrity profiles: Users with millions of followers whose profile data is fetched on every follower's feed render.
• Global configuration: Feature flags, rate limit configurations, or shared settings read by every single request.
• Flash sales: A single product page during a limited-time sale event receiving massive concentrated traffic.
• Counter keys: Global counters (like "total site visitors today") updated and read by every request.

Why Hot Keys Are Dangerous

Detecting Hot Keys

Redis Built-in Tools

# Redis MONITOR (use briefly — high overhead)
# Shows every command in real-time
MONITOR

# Redis HOTKEYS (requires LFU eviction policy)
# Redis 4.0+ with maxmemory-policy set to *lfu
redis-cli --hotkeys

# Redis OBJECT FREQ (per-key frequency)
OBJECT FREQ product:viral-item-123

# Redis slowlog — find keys causing slow operations
SLOWLOG GET 10

# Redis INFO commandstats — command frequency breakdown
INFO commandstats

Application-Level Monitoring

from collections import Counter
import time

class HotKeyDetector:
    def __init__(self, window_seconds=60, threshold=1000):
        self.window = window_seconds
        self.threshold = threshold
        self.access_log = []
        self.counter = Counter()
    
    def record_access(self, key):
        now = time.time()
        self.access_log.append((now, key))
        self.counter[key] += 1
        
        # Clean old entries
        cutoff = now - self.window
        while self.access_log and self.access_log[0][0] < cutoff:
            old_key = self.access_log.pop(0)[1]
            self.counter[old_key] -= 1
            if self.counter[old_key] <= 0:
                del self.counter[old_key]
    
    def get_hot_keys(self, top_n=10):
        return self.counter.most_common(top_n)
    
    def is_hot(self, key):
        return self.counter.get(key, 0) > self.threshold

detector = HotKeyDetector(window_seconds=60, threshold=5000)

# In your cache wrapper
def cache_get(key):
    detector.record_access(key)
    if detector.is_hot(key):
        return get_from_local_cache(key)  # Use L1 for hot keys
    return redis_client.get(key)

Solution 1: Local (L1) Cache for Hot Keys

Add an in-process local cache in front of the distributed cache. Hot keys are served from local memory with zero network overhead, distributing the load across all application instances.

from functools import lru_cache
import time

class L1L2Cache:
    def __init__(self, redis_client, l1_ttl=5, l1_max_size=1000):
        self.redis = redis_client
        self.l1_ttl = l1_ttl
        self.l1_cache = {}  # key -> (value, expiry_time)
        self.l1_max_size = l1_max_size
    
    def get(self, key):
        # L1: Check local cache first
        if key in self.l1_cache:
            value, expiry = self.l1_cache[key]
            if time.time() < expiry:
                return value  # L1 hit — zero network latency
            else:
                del self.l1_cache[key]
        
        # L2: Check Redis
        value = self.redis.get(key)
        if value is not None:
            # Promote to L1 if it looks hot
            if len(self.l1_cache) < self.l1_max_size:
                self.l1_cache[key] = (value, time.time() + self.l1_ttl)
        
        return value

Key consideration: L1 caches are per-instance, so invalidation requires a broadcast mechanism (e.g., Redis Pub/Sub) to notify all instances when hot key data changes.

Solution 2: Key Splitting (Sharding Hot Keys)

Split a single hot key into multiple sub-keys distributed across different cache nodes. Each request randomly selects one of the sub-keys, spreading the load evenly.

import random

NUM_REPLICAS = 10

def set_hot_key(key, value, ttl=300):
    """Write to all replicas"""
    for i in range(NUM_REPLICAS):
        replica_key = f"{key}:replica:{i}"
        redis_client.setex(replica_key, ttl, value)

def get_hot_key(key):
    """Read from a random replica"""
    replica_id = random.randint(0, NUM_REPLICAS - 1)
    replica_key = f"{key}:replica:{replica_id}"
    value = redis_client.get(replica_key)
    
    if value is None:
        # Fallback: try other replicas
        for i in range(NUM_REPLICAS):
            if i != replica_id:
                value = redis_client.get(f"{key}:replica:{i}")
                if value is not None:
                    return value
    
    return value

def invalidate_hot_key(key):
    """Delete all replicas"""
    pipe = redis_client.pipeline()
    for i in range(NUM_REPLICAS):
        pipe.delete(f"{key}:replica:{i}")
    pipe.execute()

With 10 replicas distributed across a Redis Cluster, the load is spread across up to 10 different nodes instead of being concentrated on one. This reduces per-node load by up to 10x.

Solution 3: Read Replicas

In Redis Cluster, each master node can have multiple read replicas. Configure your application to read hot keys from replicas, distributing read load across multiple servers while the master handles writes.

# Redis Cluster configuration for read replicas
# redis.conf on replica nodes
replica-read-only yes

# Python client with read-from-replica support
from redis.cluster import RedisCluster

rc = RedisCluster(
    host="redis-cluster.internal",
    port=6379,
    read_from_replicas=True  # Automatically route reads to replicas
)

# Reads are load-balanced across master and replicas
value = rc.get("hot:product:1001")

Solution 4: Rate Limiting per Key

Instead of trying to serve unlimited traffic for a hot key, cap the request rate and serve cached responses or queued responses beyond the limit.

def get_with_rate_limit(key, max_rps=10000):
    rate_key = f"rate:{key}:{int(time.time())}"
    current = redis_client.incr(rate_key)
    if current == 1:
        redis_client.expire(rate_key, 2)
    
    if current > max_rps:
        # Over limit — return last known value or default
        return get_stale_value(key)
    
    return redis_client.get(key)

Solution Comparison

Real-World Examples

Twitter Trending Topics: When a topic trends, millions of users view the same trending data. Twitter uses local caches on each application server with short TTLs, backed by a distributed cache. The local cache absorbs the burst; the distributed cache serves as the source of truth.

Amazon Flash Sales: During Prime Day, a small number of deal pages receive enormous traffic. Amazon pre-warms these keys across multiple cache nodes using key splitting and proactively refreshes them to prevent cache stampedes.

Gaming Leaderboards: A global leaderboard stored in a Redis Sorted Set becomes a hot key as millions of players check rankings. Solutions include snapshotting the leaderboard into read-only replicas every few seconds and serving the snapshot from L1 caches.

Prevention Strategies

• Design for uniform distribution: Use consistent hashing with virtual nodes to spread keys evenly across nodes.
• Monitor proactively: Set up alerts when any single key exceeds a request threshold. Catch hot keys before they become emergencies.
• Plan for viral scenarios: If your application could have viral content (social media, e-commerce), build hot key handling into your architecture from day one.
• Use cache-aside with L1: The L1/L2 hierarchy naturally handles hot keys because the hottest data stays in the fastest cache.

Frequently Asked Questions

How do I know if I have a hot key problem?

Symptoms include: one Redis node consistently has higher CPU/network usage than others, latency spikes that correlate with traffic to specific content, evictions on one node while others have free memory, and redis-cli --hotkeys showing extreme outliers. Application-level monitoring that tracks per-key access counts is the most reliable detection method.

What is the L1 TTL for hot keys?

Keep L1 TTLs very short — typically 1-10 seconds. The L1 cache is meant to absorb bursts, not provide long-term caching. A 5-second L1 TTL on 20 application servers means each server refreshes from Redis every 5 seconds (4 requests/second per server, 80 total) instead of every single user request hitting Redis (potentially thousands per second per server). Even a 1-second L1 TTL provides massive relief.

Does key splitting work with Redis Cluster?

Yes, and it works particularly well because different replica keys will map to different hash slots, naturally distributing across different nodes. Avoid using hash tags for split keys — you want them on different nodes. For example, product:1001:replica:0 through product:1001:replica:9 will likely land on different hash slots across the cluster.

Can hot keys cause data loss?

Indirectly, yes. An overloaded node may become unresponsive, triggering failover. During failover, asynchronous replication lag means some recent writes may be lost. Additionally, if the hot key triggers stampede behavior on the database after a node failure, the database itself may become overwhelmed, leading to broader data access failures.