Twitter (X): fan-out for home timelines
The classic home timeline design: do the work when someone posts, so the far more frequent reads are cheap.
The idea
When reads vastly outnumber writes, build the timeline at write time. Twitter's timeline service did: a new post is inserted into the cached timeline of each follower, so reading a timeline is a single lookup. Timelines hold IDs rather than posts, and are capped in length, so the cache stays small.
The design has a well-known weak spot: accounts with enormous followings, where one post becomes millions of inserts. The talk covers that problem and the hybrid answer, Fan-out on write and fan-out on read on write for most accounts and a merge at read time for the largest. It also treats timelines as derived data that can be rebuilt, which is why users who have not visited in a while can be skipped entirely.
Read the originals
Written by the engineers who built it.
- Timelines at Scale
Raffi Krikorian · Talk, Apr 2013
A talk on how home timelines were served: fan-out on write into a cache, what that costs for very large accounts, and why timelines are treated as rebuildable.
Practise it
Make the decisions yourself, then compare.