Design a Distributed Cache (Memcache), stage 8 of 9: break it
A cluster with an empty cache
Other clusters in the region have warm caches with roughly the same data.
System so far· 7 parts
Select a component to see what it is responsible for and which state it owns.
- 1Users → Web servers: Page request
- 2Web servers → mcrouter: get / multiget, delete
- 3mcrouter → memcached pool: Keys by consistent hash
- 4Web servers → MySQL: Query on miss; writes
- 5MySQL → Invalidation daemon: Committed deletes
- 6Invalidation daemon → mcrouter: Batched deletes
- 7mcrouter → Gutter pool: On server failure
- Request / response
- Bulk data
- Asynchronous
What you need to know
0 of 2 checks done
Work it out
A cluster serves 2,000,000 reads a second. Warm, its hit rate is 99%. Cold, it's about 5%. About how many database reads a second does it send when cold?Other clusters in the region hold roughly the same data, warm. A cold cluster can treat a neighbour's cache as its fallback: on a local miss, fetch from the warm cluster's cache and add it locally, so misses come from memory instead of the database.
Think first
A key is written and deleted in both clusters, but the cold cluster receives its delete slightly before the warm one. In that gap, a cold-cluster miss fetches the key from the warm cluster. What happens?