Chat App

A chat app looks simple — send a message, the other person sees it instantly — but it's one of the best interview problems because it forces you to confront stateful connections, cross-server routing, and storing a huge, ever-growing stream of messages. Let's build it with the Real-time Updates pattern at the core.

1. Requirements

Functional:

• One-to-one and group messages, delivered in real time.
• Message history (scroll back).
• Delivery/read receipts and online presence (nice-to-have).

Non-functional:

• Real-time — low latency, two-way.
• Scale — millions of concurrent connected users; message volume is a firehose.
• Durable — messages must not be lost; history must persist.
• Ordered — within a conversation, messages arrive in order.

2. The transport: WebSockets

Chat is two-way and high-frequency, so WebSockets are the right transport — a persistent duplex connection between each client and a server. Polling would be wasteful; SSE is one-directional. The client opens a socket on login and keeps it open.

3. The core challenge: connections are stateful

Here's the problem that shapes the whole design. User A is connected to server 1; user B to server 2. A sends a message for B — but it arrives at server 1, which doesn't hold B's socket. How does it reach server 2?

You need two things:

• A connection registry: who is connected, and to which server. Store user → server mapping (in Redis), updated on connect/disconnect.
• A way to route a message to the right server. When server 1 gets a message for B, it looks up B's server and forwards it — commonly by publishing to a Redis pub/sub channel that B's server subscribes to. Server 2 receives it and pushes it down B's socket.

A ─► server 1 ─► (store message) ─► publish to channel for B
                                    server 2 (holds B's socket) ─► push to B

This pub/sub fan-out is the heart of a scalable chat backend: any server can deliver to any user without knowing where their socket lives.

4. Storing messages

Message history is append-heavy and enormous — a classic fit for a wide-column store keyed by conversation:

messages_by_conversation (
  conversation_id   ← partition key (a chat's messages live together)
  sent_at           ← clustering column (ordered)
  message_id, sender_id, body
)

"Load the last 50 messages in this chat" becomes one fast, pre-sorted read of one partition. Writes are cheap appends. At smaller scale, Postgres with an index on (conversation_id, sent_at) is perfectly fine — don't reach for Cassandra until the volume demands it.

5. Delivery and offline users

• Write first, then deliver. Persist the message before pushing it, so it survives even if the recipient is offline — they get it from history on next connect.
• Offline recipients: the message is stored; when B reconnects, they fetch anything since their last-seen. A notification (push/APNs) tells them a message is waiting.
• Receipts: "delivered" and "read" are small status updates sent back over the socket and persisted.
• Ordering: a per-conversation sequence or timestamp keeps messages ordered; the client sorts by it.

6. Group chat and fan-out

A group message must reach every member. Small groups: look up each member's server and deliver via pub/sub, same as 1:1. Large groups/broadcast (thousands of members) is a fan-out decision — you may write once and let each member's connection pull, rather than pushing thousands of copies synchronously.

7. Deep dives

• Scaling connections: millions of sockets means many servers each holding a slice; a load balancer that supports long-lived connections distributes them, and you must drain gracefully on deploy.
• Presence: "online" is a Redis key with a short TTL, refreshed by a heartbeat over the socket; its absence means offline. Presence at huge scale is genuinely hard (every status change can fan out to many watchers) — worth calling out.
• Reconnection: clients must reconnect and resync missed messages after a drop; design the "messages since X" fetch for it.

What to emphasise

Lead with the transport, then the cross-server routing that makes it scale:

"Each client holds a WebSocket to one of many servers. The hard part is that sender and recipient are usually on different servers, so I keep a user → server registry in Redis and route messages via Redis pub/sub — any server can deliver to any user's socket. Messages are persisted first (in a store keyed by conversation, so history is one fast read) then pushed, so offline users get them from history plus a push notification. Presence is a heartbeat-refreshed Redis key with a TTL."

Next up: News Feed — the fan-out problem in full.