Skip to content
sysgeeks

Design Discord's Message Storage, stage 8 of 9: break it

Write the coalescer

Implement the read path in the data service so that identical concurrent reads share one database query. Permissions have already been checked by the API servers. Think about what happens when the shared query fails or hangs.

System so far· 5 parts
12345CLIENTMembersSERVICEAPI serversSERVICEGatewaySERVICEMessagedata serviceDATABASEMessage cluster

Select a component to see what it is responsible for and which state it owns.

  1. 1Members → API servers: Send, load history, jump
  2. 2API servers → Gateway: New message event
  3. 3Gateway → Members: Push to online members
  4. 4API servers → Message data service: Query by channel (hash-routed)
  5. 5Message data service → Message cluster: Read and write one partition
  • Request / response
  • Asynchronous
  • Server push

What you need to know

0 of 2 checks done

  1. A coalescer is a map from request key to in-flight promise:

    1. If the key is in the map, await the existing promise.
    2. Otherwise, start the query, store its promise, and remove the entry when it settles.

    The key must include every parameter that changes the result: channel, cursor and limit. Two requests that differ in any of them aren't the same question.

  2. Check

    The shared query fails. What must happen to its map entry?