Skip to content
sysgeeks

Design a News Feed (Twitter Timeline), stage 9 of 9: defend it

Defend the hybrid

Your interviewer: "You have two read paths, a queue doing a third of a million inserts a second, and terabytes of Redis. Wouldn't fan-out on read with a good cache be simpler?"

System so far· 9 parts
123456789CLIENTAuthorsSERVICEWrite APIDATABASEPost storeQUEUEFan-out queueWORKERFan-out workersSERVICESocial graphCACHETimeline cacheSERVICETimeline serviceCLIENTReaders

Select a component to see what it is responsible for and which state it owns.

  1. 1Authors → Write API: Post
  2. 2Write API → Post store: Store post
  3. 3Write API → Fan-out queue: Fan-out job
  4. 4Fan-out workers → Fan-out queue: Take jobs
  5. 5Fan-out workers → Social graph: Active followers
  6. 6Fan-out workers → Timeline cache: Push ID, trim to 800
  7. 7Readers → Timeline service: GET home timeline
  8. 8Timeline service → Timeline cache: Page of IDs
  9. 9Timeline service → Post store: Hydrate; large accounts' recent posts
  • Request / response
  • Asynchronous

What you need to know

0 of 1 checks done

  1. "Wouldn't X be simpler?" deserves a numeric answer. Restate the ratio that drove the design, show what the alternative costs in the same units, concede the real costs of yours, and say what would change your mind.

  2. Check

    "Use pull with a good cache." Why doesn't caching rescue pull here?