Skip to content
sysgeeks

Design a News Feed (Twitter Timeline), stage 7 of 9: break it

Write the merged read

Implement the read path for the hybrid design: the reader's precomputed list, plus the recent posts of any very large accounts they follow, merged newest first. Post IDs are time-sortable, so a larger ID is a newer post. Support a before cursor for the next page.

System so far· 9 parts
123456789CLIENTAuthorsSERVICEWrite APIDATABASEPost storeQUEUEFan-out queueWORKERFan-out workersSERVICESocial graphCACHETimeline cacheSERVICETimeline serviceCLIENTReaders

Select a component to see what it is responsible for and which state it owns.

  1. 1Authors → Write API: Post
  2. 2Write API → Post store: Store post
  3. 3Write API → Fan-out queue: Fan-out job
  4. 4Fan-out workers → Fan-out queue: Take jobs
  5. 5Fan-out workers → Social graph: Active followers
  6. 6Fan-out workers → Timeline cache: Push ID, trim to 800
  7. 7Readers → Timeline service: GET home timeline
  8. 8Timeline service → Timeline cache: Page of IDs
  9. 9Timeline service → Post store: Hydrate; large accounts' recent posts
  • Request / response
  • Asynchronous

What you need to know

0 of 2 checks done

  1. Post IDs here are time-sortable: a larger ID is a newer post (Twitter's Snowflake IDs start with a timestamp). That makes merging sources a sort by ID, and pagination a cursor: "give me posts with ID less than X". See Generating unique identifiers.

  2. Check

    Two 64-bit IDs as decimal strings: "999" and "1000". Compared as strings, which is 'larger'?