Scaling Memcache at Facebook
Rajesh Nishtala, Hans Fugal, Steven Grimm, et al.2013NSDI 2013
Read it on usenix.org(opens in a new tab)Why this one
Most engineers first meet memcache as a hashmap with TTLs. Facebook had to treat it as a living distributed system, with routing, invalidation, thundering herds, regional failure, cold starts, and operator mistakes all in the design. The useful idea here is not that caching makes reads faster. It is that a cache layer becomes part of your consistency model, your deploy model, and your incident model once enough product paths depend on it. People get this wrong by arguing about hit rate in isolation, then discovering that misses, evictions, and invalidations are where the outages hide. Read this after Consistent Hashing and Random Trees, then pair it with The Tail at Scale. One gives you placement, the other gives you latency pain, and this shows what a real cache fleet looks like when both meet production.
What to take away
- Cache invalidation is a distributed write path, not a background cleanup task.
- A high hit rate can still fail users if cold keys stampede the database at once.
- Regional pools, leases, and replication are used to bound blast radius, not just improve speed.
Reads with
- Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web
gives the placement primitive behind sharding hot cache traffic without central coordination
- The Tail at Scale
explains why cache misses and fanout turn rare latency into user-visible pain
- Dynamo: Amazon's Highly Available Key-value Store
shows the storage-side availability tradeoffs that memcache pushes up into the cache layer