Skip to content
sysgeeks

Design Discord's Message Storage, stage 3 of 9: decide

Choose the partition key

In this store, the partition key decides which nodes hold a row and which rows are stored together. The clustering key orders rows inside a partition. Some channels will receive millions of messages over their lifetime; most receive a few hundred.

System so far· 4 parts
1234CLIENTMembersSERVICEAPI serversSERVICEGatewaySERVICEMessagedata service

Select a component to see what it is responsible for and which state it owns.

  1. 1Members → API servers: Send, load history, jump
  2. 2API servers → Gateway: New message event
  3. 3Gateway → Members: Push to online members
  4. 4API servers → Message data service: Query by channel (hash-routed)
  • Request / response
  • Asynchronous
  • Server push

What you need to know

0 of 2 checks done

  1. Partitions must stay bounded. A partition that grows forever (many gigabytes) makes compaction, repair and replacing a node slow and memory-hungry, and reads of it get slower.

    The natural key here, the channel, is unbounded: a busy channel gets messages for years.

  2. Bucketing bounds it: add a time window to the partition key, say 10 days. Partition = (channel, bucket), where the bucket is computed from the message's timestamp. Each partition holds at most 10 days of one channel.

    Because the bucket comes from the Snowflake ID's timestamp, any message's partition can be computed from its ID alone.

  3. Work it out

    A very busy channel gets 10,000 messages a day at 1 KB each. About how many megabytes in one 10-day bucket?
    MB