Caching Strategies at Scale: Beyond Simple Key-Value Stores

Introduction

Caching is the backbone of every high-performance backend system, but naive caching leads to stale data, cache stampedes, and consistency nightmares. Let us explore patterns that work at scale.

Caching Patterns

There are four fundamental caching strategies, each with distinct trade-offs for consistency, latency, and complexity.

Cache-Aside (Lazy Loading)

The application is responsible for reading from and writing to the cache. On a cache miss, it reads from the database and populates the cache.

The application is responsible for reading from and writing to the cache. On a cache miss, it reads from the database and populates the cache.

sequenceDiagram
    participant App
    participant Cache
    participant DB

    App->>Cache: Get(Key)
    alt Cache Hit
        Cache-->>App: Value
    else Cache Miss
        Cache-->>App: nil
        App->>DB: Query()
        DB-->>App: Result
        App->>Cache: Set(Key, Result)
    end

func GetUser(ctx context.Context, id string) (*User, error) {
    // 1. Check cache first
    cached, err := cache.Get(ctx, "user:"+id)
    if err == nil {
        return deserialize(cached)
    }
    // 2. Cache miss -- fetch from database
    user, err := db.GetUser(ctx, id)
    if err != nil {
        return nil, err
    }
    // 3. Populate cache
    cache.Set(ctx, "user:"+id, serialize(user), 5*time.Minute)
    return user, nil
}

This is the most common pattern and works well for read-heavy workloads. The main drawback is that the first request after a cache miss or expiration is slow.

Write-Through

Every write goes to both the cache and the database synchronously. This ensures the cache is always consistent with the database but adds latency to writes.

Write-Behind (Write-Back)

Writes go to the cache immediately and are asynchronously flushed to the database. This provides the fastest write performance but risks data loss if the cache fails before flushing.

sequenceDiagram
    participant App
    participant Cache
    participant Queue as Msg Queue
    participant DB
    
    Note over App: Client Request
    App->>Cache: Set(Key, Value)
    App->>Queue: Publish(WriteEvent)
    App-->>App: Return 200 OK (Fast)
    
    loop Async Worker
        Queue->>DB: Batch Write
    end

// Write-Behind Implementation (Simplified)
func UpdateUserProfile(ctx context.Context, user User) error {
    // 1. Update Cache Immediately (Optimistic)
    err := cache.Set(ctx, "user:"+user.ID, user, 0)
    if err != nil {
        return err
    }
    
    // 2. Queue for async persistence
    // This returns almost instantly
    err = queue.Publish("db_writes", WriteEvent{
        Type: "UPDATE_USER",
        Data: user,
    })
    return err
}

Read-Through

Similar to cache-aside, but the cache itself is responsible for loading data on a miss. The application only talks to the cache, which talks to the database when needed.

Cache Stampede Prevention

A cache stampede occurs when a popular cache key expires and hundreds of requests simultaneously hit the database to reload it. There are several mitigation strategies.

func GetUserWithLock(ctx context.Context, id string) (*User, error) {
    cached, err := cache.Get(ctx, "user:"+id)
    if err == nil {
        return deserialize(cached)
    }
 
    // Acquire a distributed lock
    lockKey := "lock:user:" + id
    acquired, err := cache.SetNX(ctx, lockKey, "1", 10*time.Second)
    if err != nil || !acquired {
        // Another request is loading, wait and retry
        time.Sleep(100 * time.Millisecond)
        return GetUserWithLock(ctx, id)
    }
    defer cache.Del(ctx, lockKey)
 
    // Double-check cache (another request may have loaded it)
    cached, err = cache.Get(ctx, "user:"+id)
    if err == nil {
        return deserialize(cached)
    }
 
    user, err := db.GetUser(ctx, id)
    if err != nil {
        return nil, err
    }
    cache.Set(ctx, "user:"+id, serialize(user), 5*time.Minute)
    return user, nil
}

Redis Cluster for High Throughput

Single Redis instance tops out at roughly 100K operations per second. Redis Cluster shards data across multiple nodes using hash slots, providing linear scalability.

graph TD
    Client[Application]
    
    subgraph Cluster [Redis Cluster]
        N1["Node 1<br/>Slots 0-5460"]
        N2["Node 2<br/>Slots 5461-10922"]
        N3["Node 3<br/>Slots 10923-16383"]
    end

    Client -->|"CRC16 key mod 16384"| Hash{"Hash Slot"}
    Hash -->|"0-5460"| N1
    Hash -->|"5461-10922"| N2
    Hash -->|"10923-16383"| N3

rdb := redis.NewClusterClient(&redis.ClusterOptions{
    Addrs: []string{
        "redis-1:6379",
        "redis-2:6379",
        "redis-3:6379",
    },
    PoolSize:     100,
    MinIdleConns: 10,
    ReadOnly:     true,
    RouteByLatency: true,
})

Conclusion

Effective caching requires understanding your data access patterns, consistency requirements, and failure modes. Choose the right pattern for each use case and always plan for cache failures. A cache should improve performance, not become a single point of failure.