hard general · part of Practice Questions · Senior SWE Roadmap
Requirements to clarify
- Functional: get/set/delete a key, with reasonable latency guarantees.
- Non-functional: extremely low latency (sub-millisecond), high throughput, data may be larger than one machine’s memory (needs to be distributed), how important is durability vs pure speed.
Core components
- Partitioning: consistent hashing distributes keys across cache nodes, minimizing remapping when nodes are added/removed (see Consistent Hashing).
- Eviction: LRU/LFU policy per node to bound memory usage (see Caching Strategies).
- Replication: each partition replicated to a couple of nodes for availability if a cache node dies (a cache miss cascading to the DB is much cheaper to tolerate than a full outage).
- Client-side routing: clients (or a routing layer) hash the key and go directly to the owning node, avoiding an extra hop.
Key tradeoffs
- Pure in-memory (fast, data lost on restart) vs persisted-to-disk cache (survives restarts, some latency cost) — depends on whether the cache is a pure accelerator (source of truth elsewhere) or something closer to a primary store.