A chat or notification system is a genuinely good vehicle for a design interview specifically because it forces queues, caching, and scaling decisions to interact with each other in the same design, rather than appearing as isolated, independent topics the way they might in a simpler CRUD system.
Functional: a user sends a message (in a 1:1 chat, a group chat, or a broadcast notification), and it needs to reach the recipient(s) — either delivered live if they're online, or waiting for them the next time they open the app. Non-functional: low latency for online users, no message loss even if a recipient is offline or a server crashes mid-delivery, and the design has to hold up for both a small group chat and a "notify everyone" broadcast to millions of users, which turn out to need different strategies entirely.
This is the central design decision, and it's a direct trade-off between write cost and read cost:
| Fan-out on write | Fan-out on read | |
|---|---|---|
| Write cost | Scales with recipient count — expensive for huge audiences | Constant — one write regardless of audience size |
| Read cost | Cheap — read your own pre-built inbox | More expensive — assembled per read |
| Good fit | Normal users with a bounded, everyday-sized audience (a 1:1 chat, a small group) | Very large broadcast audiences, where per-recipient fan-out at send time would be enormous |
Most real systems that need both use a hybrid: fan-out on write for the common case (a message with a normal-sized audience gets pushed to recipients' inboxes immediately, keeping reads fast for the overwhelming majority of traffic), with a fallback to fan-out on read specifically for exceptionally large audiences, where paying a million writes at send time for one message would itself be the bottleneck. This hybrid split is exactly the kind of trade-off-naming step 5 of the framework is asking for — neither pure strategy is right for both cases, and saying so explicitly, with the reasoning, is the actual signal an interviewer is looking for.
Delivering a message isn't one atomic step — it typically means: persist the message durably, fan it out to recipient inboxes, and push it live to any recipient currently connected. Doing all three synchronously in the request path makes the sender wait on work that doesn't need to block them; instead, the send path durably persists the message and then drops a fan-out job onto a message queue (chapter 7's pub/sub pattern fits naturally — a message-sent event that the fan-out workers and the live-push service both subscribe to independently), letting the sender's request return immediately once the message is safely stored.
A cache earns its place on the read side the same way it does everywhere else in this reference: a chat client re-opening a conversation needs its most recent messages fast, and those are exactly the ones most likely to be requested repeatedly in a short window — a natural fit for cache-aside in front of the message-history database, while older history a user scrolls back to rarely stays cache-resident and is fine falling through to the database directly.
A connected client typically holds a persistent connection (WebSocket, or a comparable long-lived channel) to a specific server instance, and that server pushes new messages down that connection the moment they arrive — but this reintroduces the statelessness tension from chapter 2 in yet another form: "which server is this specific user currently connected to" is now state that has to live somewhere every other server can look it up (again, commonly a shared store like Redis, mapping user ID to the server instance holding their live connection), so a message fanned out from any app server can find the one server that needs to push it down an open socket.
An offline recipient simply has no open connection to push to — their fanned-out copy sits in their inbox exactly as written, and the moment they reconnect, a normal read against that inbox (warmed by the same cache-aside pattern) picks it up with no special-cased "offline delivery" logic needed at all. That's a deliberately elegant property of the fan-out-on-write design: online push and offline persistence aren't two different systems, they're one write path with two different consumers of the same stored result.