Caching Strategies

Why caching exists

A cache trades a small amount of (fast, expensive) storage for a large reduction in latency and load on a slower system of record. Three things make a piece of data worth caching:

Read-heavy: read far more often than written.
Expensive to (re)compute or fetch: a slow query, a join across tables, an external API call.
Tolerant of some staleness: the consumer doesn’t need the absolute latest value on every read.

If a value is written as often as it’s read, or must always be perfectly fresh, caching adds complexity without much payoff — that’s the first thing to check before reaching for a cache in an interview.

Where caches live

Caching happens at every layer between the user and the source of truth. Each layer trades latency for hit rate and freshness differently.

flowchart TD
    Client["Browser cache<br/>(HTTP cache headers)"] --> CDN["CDN / edge cache<br/>(static assets, API responses)"]
    CDN --> LB[Load Balancer]
    LB --> App1[App Server 1]
    LB --> App2[App Server 2]
    App1 --> AppCache[("Distributed cache<br/>Redis / Memcached")]
    App2 --> AppCache
    App1 -. "local in-process cache<br/>(hot keys, sub-ms)" .-> App1
    AppCache --> DBCache[("DB query cache /<br/>buffer pool")]
    DBCache --> DB[(Primary Database)]

Browser / client cache: driven by HTTP headers (Cache-Control, ETag); zero network cost on a hit.
CDN / edge: caches static assets and sometimes whole API responses close to the user (see CDN & Content Delivery).
In-process (local) cache: a map inside the app server’s own memory — nothing beats it for latency, but it’s per-instance (N servers = N copies, no sharing) and disappears on restart.
Distributed cache: a shared cache tier (Redis, Memcached) that every app server hits — one copy, consistent across instances, survives individual app restarts, but is a network hop away.
Database-level cache: the DB’s own buffer pool / query cache — the last line of defense before hitting disk.

A common senior-level move: multi-level caching — a small local cache in front of the distributed cache, to absorb extremely hot keys without even paying the network round-trip. The cost is an extra layer of staleness to reason about.

Access patterns (the actual “strategies”)

These describe how the cache sits relative to reads and writes.

Cache-aside (lazy loading)

The application owns the logic: check the cache first, fall back to the DB on a miss, then populate the cache for next time.

sequenceDiagram
    participant C as Client
    participant A as App Server
    participant Ca as Cache
    participant D as Database

    C->>A: GET /item/42
    A->>Ca: GET item:42
    alt Cache hit
        Ca-->>A: value
        A-->>C: 200 OK (from cache)
    else Cache miss
        Ca-->>A: nil
        A->>D: SELECT * FROM items WHERE id=42
        D-->>A: row
        A->>Ca: SET item:42, value, TTL
        A-->>C: 200 OK (from DB)
    end

Pros: only requested data gets cached (no wasted space); cache going down just means falling back to the DB (fail-safe).
Cons: every miss pays the full DB round-trip and the cache-populate step; the first request after any eviction is always slow.
The most common pattern in practice — this is what “put Redis in front of your DB” usually means.

Read-through

Functionally similar to cache-aside, but the cache itself (not the application) is responsible for loading from the DB on a miss — the app only ever talks to the cache. Common in caching libraries/frameworks that wrap the data access layer. Same performance characteristics as cache-aside; the difference is purely where the “on miss, go load it” logic lives.

Write-through

Every write goes to the cache, and the cache synchronously writes it to the DB before acknowledging.

sequenceDiagram
    participant App
    participant Cache
    participant DB

    App->>Cache: write(key, value)
    Cache->>DB: write(key, value)
    DB-->>Cache: ack
    Cache-->>App: ack

Pros: cache is always consistent with the DB — no stale-cache window after a write.
Cons: every write pays cache and DB latency; data that’s written but never read still gets cached (can waste space).

Write-behind (write-back)

Writes land in the cache and are acknowledged immediately; the cache flushes to the DB asynchronously (batched, on a timer, or on eviction).

sequenceDiagram
    participant App
    participant Cache
    participant DB

    App->>Cache: write(key, value)
    Cache-->>App: ack (immediate)
    Note over Cache: value sits in cache
    Cache--)DB: async batched flush

Pros: fastest possible writes; can batch/coalesce many writes into fewer DB operations.
Cons: a cache crash before flush loses data — only acceptable when some write loss is tolerable, or the cache itself is made durable (e.g., AOF/RDB persistence in Redis).

Write-around

Writes go straight to the DB, bypassing the cache entirely; the cache only gets populated on a subsequent read (via cache-aside). Good for data that’s written often but rarely read right after — avoids filling the cache with write-only churn.

Comparison

Pattern	Write latency	Read latency (after write)	Risk
Cache-aside	N/A (writes go to DB directly, cache invalidated)	Slow on first read after miss	Stale cache if invalidation is missed
Read-through	N/A	Same as cache-aside	Same
Write-through	Slow (cache + DB)	Fast (always warm)	None on freshness, costs write latency
Write-behind	Fast	Fast	Data loss if cache dies before flush
Write-around	Fast (DB only)	Slow on first read	Cache never warmed by writers

Eviction policies

A cache has finite capacity; something has to be removed when it’s full.

LRU (Least Recently Used) — evict whatever hasn’t been accessed in the longest time. Implemented as a hash map (key → node) + doubly linked list (access order), giving O(1) get/put/evict. The most common default — recency is usually a good proxy for “will be needed again.”
LFU (Least Frequently Used) — evict the lowest access-count item. Better than LRU when popularity is stable over time, but slower to adapt when access patterns shift, and needs extra bookkeeping (counters, or a frequency-bucketed structure) to stay O(1).
FIFO — evict in insertion order, ignoring access pattern entirely. Simple, rarely optimal, but cheap.
TTL / expiration-based — every entry has a lifetime; expires regardless of access pattern. Usually combined with one of the above (TTL bounds staleness, LRU/LFU decides what to evict early under memory pressure).
Random — evict a random entry. Surprisingly competitive with LRU under some workloads and trivially O(1) with no bookkeeping (Redis supports this as allkeys-random).

flowchart LR
    Head["MRU (head)<br/>just accessed"] --> A["key: B"]
    A --> B["key: A"]
    B --> Tail["LRU (tail)<br/>evicted first when full"]

Interview framing: LRU is the safe default to name first; be ready to explain why you’d switch to LFU (stable popularity skew, e.g. a small set of celebrity users) or add a TTL (bound staleness regardless of access pattern).

Cache invalidation

“There are only two hard things in computer science: cache invalidation and naming things.”

The core risk of any cache is serving data that’s no longer true. Strategies, roughly cheapest to most precise:

TTL expiration: simplest — accept up to N seconds of staleness, no explicit invalidation logic needed. Good default when the business can tolerate a bounded staleness window.
Explicit invalidation on write: the write path actively deletes or updates the corresponding cache key(s). Precise, but easy to miss a code path that writes without invalidating (a classic source of stale-cache bugs).
Versioned / keyed entries: bake a version (or the underlying row’s updated_at) into the cache key itself (e.g. user:42:v7); a write bumps the version instead of hunting down and deleting the old key. Old versions simply age out via normal eviction — nothing to explicitly delete, and there’s no window where a stale key is served under the “current” name.
Event-based invalidation: writes publish an event (see Message Queues & Event-Driven Architecture); cache nodes subscribe and invalidate on receipt. Needed once more than one process can write to the same underlying data — a single app’s local write hook isn’t enough to invalidate copies in other processes’ local caches.

Cache stampede (thundering herd)

A popular key expires, and a burst of concurrent requests all miss at once, all hit the DB at once — a self-inflicted spike that can take down the very database the cache was protecting.

sequenceDiagram
    participant C1 as Client 1
    participant C2 as Client 2
    participant C3 as Client 3
    participant Ca as Cache
    participant D as Database

    Note over Ca: hot key just expired
    C1->>Ca: GET key (miss)
    C2->>Ca: GET key (miss)
    C3->>Ca: GET key (miss)
    C1->>D: expensive query
    C2->>D: expensive query
    C3->>D: expensive query
    Note over D: DB hit 3x simultaneously for the same data

Fixes:

Mutex / single-flight locking: the first request to miss acquires a lock and does the DB fetch; concurrent requests for the same key either wait briefly for the result or get served the (slightly) stale value while the fetch is in flight, instead of all hitting the DB.

sequenceDiagram
    participant C1 as Client 1
    participant C2 as Client 2
    participant Ca as Cache
    participant D as Database

    C1->>Ca: GET key (miss)
    Ca-->>C1: acquire lock: granted
    C2->>Ca: GET key (miss)
    Ca-->>C2: lock held — wait or serve stale
    C1->>D: expensive query
    D-->>C1: result
    C1->>Ca: SET key + release lock
    Ca-->>C2: fresh value now available

Probabilistic early expiration: recompute slightly before actual expiry, with randomized jitter per key, so many keys with the same TTL don’t all expire in the same instant.
Stale-while-revalidate: serve the stale value immediately while asynchronously refreshing it in the background — the user never waits, and only one background refresh happens.
Never expire under load, refresh proactively: for very hot keys, have a background job keep the value fresh on a schedule instead of relying on TTL expiry + reactive reload at all.

Hot keys

A related but distinct problem: in a distributed cache, one key can get so much traffic that the single node/shard owning it becomes a bottleneck — even though the cache overall has plenty of headroom (see Consistent Hashing for how keys map to nodes). Fixes: replicate the hot key across multiple nodes and pick one at random per read, or add a small local (in-process) cache in front of the distributed tier specifically for detected hot keys.

Consistency & staleness

Strong consistency (write-through): reads always see the latest write, at the cost of write latency. Rare to need this for a cache specifically — if you truly need strong consistency, that’s usually an argument for reading from the DB directly for that path.
Eventual consistency (cache-aside, write-behind, TTL-based): reads may lag writes by some bounded window. This is the default assumption for most caching — the interview-winning move is stating how stale (“up to 30s, bounded by TTL”) rather than leaving it undefined.
Read-your-own-writes: a user expects to immediately see their own update, even under an otherwise eventually-consistent cache. Common fix: after a write, either invalidate-then-read-through, or route that user’s next read to the DB/leader briefly.

Real-world systems

Redis: in-memory, supports rich data structures (not just strings), optional persistence (RDB snapshots / AOF log) for durability, native support for several eviction policies (allkeys-lru, allkeys-lfu, volatile-ttl, etc.).
Memcached: simpler, pure in-memory key-value, multithreaded, historically the default choice before Redis’s data structures made it more broadly useful — still a reasonable pick for a pure, minimal LRU cache.
CDN caching: same cache-aside idea at a different layer — edge node misses, pulls from origin, caches for subsequent requests from that region (see CDN & Content Delivery).
HTTP caching: Cache-Control: max-age=… (TTL-based), ETag / If-None-Match (validation — “has this changed since I last saw it?”, avoids re-downloading unchanged data even after the local copy is technically stale).

Interview angles

Caching comes up in nearly every system design answer — know which access pattern to reach for by default (cache-aside) and be ready to justify a deviation (write-through for a system that can’t tolerate any stale-read window, write-behind for a write-heavy system that can tolerate some loss risk).
“What happens when the cache and DB disagree?” → talk through invalidation strategy and the staleness window it implies.
“This key is really hot, what happens?” → hot key problem, not just general cache sizing.
“Your cache just went down, what happens to the system?” → cache-aside degrades gracefully (fall through to DB, slower but correct); a system that requires the cache to be up is a design smell worth calling out.
A strong answer states the staleness bound explicitly (“reads may lag writes by up to the TTL, N seconds”) rather than treating “eventually consistent” as a hand-wave.

Notes

Explorer