Design a Notification System, stage 2 of 11: decide
Decouple from the product action
Someone comments on an issue. Three followers should be notified. The comment must save quickly and reliably whatever the email provider is doing.
System so far· 3 parts
Select a component to see what it is responsible for and which state it owns.
- 1Product services → Event stream: Domain events via outbox
- 2Event stream → Notification planner: Events, at least once
What you need to know
When a request does two things, the slower and less reliable one sets the pace for both. If saving a comment also sends emails, the comment is as slow as the email provider and fails whenever the provider fails.
The fix is to make the comment record that notifications are owed, and let something else do the sending later.
Think first
After saving the comment, the server starts an in-memory background task to send the emails, then responds. A deploy restarts the server 50 ms later. What happens?A dual write updates two systems separately, say the database and then a queue. A crash between the two leaves them disagreeing: a comment with no notification, or a notification for a comment that was rolled back.
The transactional outbox avoids it: write the comment and an event row ("comment 991 created") in the same database transaction. Either both exist or neither does. A relay later reads new outbox rows and publishes them. See Transactional outbox.
Check
With an outbox, the comment and its event commit together. The relay crashes before publishing the event. What happens?