System Design · Architecture under constraints
How would you design a chat system?
Short answer
Use persistent connections for delivery, store messages per conversation with a monotonic sequence, fan out to online devices, and queue for offline users. Delivery, read receipts and ordering need separate guarantees.
Assume one-to-one and group chat, online and offline delivery, multi-device sync, read receipts and message history. Target low latency for online users, durable messages, and per-conversation ordering. A user should not see a message twice after a reconnect.
Core components
- WebSocket or long-lived connection gateway for real-time delivery
- Presence service tracking which devices are connected
- Message service that assigns a per-conversation sequence number and persists the message
- Fan-out to recipient devices through the connection gateway or a pub-sub bus
- Offline inbox or per-device cursor for messages missed while disconnected
- Object storage for attachments with metadata in the message store
Ordering is per conversation, not global. Assigning a sequence number inside a partition owned by the conversation gives a total order for that chat. The client can then detect gaps and request history. A timestamp alone is unsafe because clocks differ across servers.
Delivery has several states: accepted by the server, delivered to a device, and read by the user. Keep them separate. A message may be stored once but delivered to five devices, and a read receipt may apply to one conversation. Use idempotency keys from the client so a retry after a dropped connection does not create a duplicate message.
Fan-out cost grows with group size. For small groups, write to each member's inbox. For very large channels, use a pub-sub topic and let each online gateway subscribe, then read history from the message store. Offline users get a cursor and fetch missed messages when they reconnect.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1How do you guarantee ordering within a conversation?
- 2How does a client recover messages after a long disconnect?
- 3What changes for a group with 100,000 members?
- 4How do you prevent duplicate messages on retry?
Say it in the interview
Chat needs a persistent connection for real-time delivery, a durable message store with a per-conversation sequence, and a fan-out path to each online device. I keep accepted, delivered and read as separate states, and clients use idempotency keys so reconnects do not duplicate messages. Offline devices resume from a cursor and fetch missed history. Large channels switch from writing every inbox to pub-sub fan-out so the cost stays manageable.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live System Design mock interview and get scored feedback on depth, clarity and confidence.