System Design Drill Day 15: Real-Time Chat and Live Order Tracking

A buyer sends a merchant a message from a phone on a patchy mobile connection and then walks into a lift. The merchant is connected to a different WebSocket server from the buyer. The message still has to reach the merchant once, in the right order.

"Design a chat app" is a classic interview question. The WebSocket is the easy part. Keeping messages durable and ordered across several servers is where it gets hard. The design here is an online marketplace's buyer-merchant chat and live order tracking, on Laravel Reverb, MySQL and Redis.

Core concepts

How the server pushes to the client

There are four options, roughly in order of how live they feel. Short polling asks the server every few seconds, which is simple and wasteful. Long polling holds a request open until there's news and makes a decent fallback. SSE (Server-Sent Events) is a one-way server-to-client stream over plain HTTP that reconnects on its own, well suited to status feeds. WebSockets keep a persistent two-way connection, which is what chat needs.

Connections hold state

Buyer A's socket lives on WebSocket node 1 and merchant B's on node 3. For a message to cross between them, every node subscribes to a shared pub/sub layer, here Redis pub/sub. Any server publishes, and whichever node holds the recipient's socket delivers.

Keep the API and the socket gateway separate. Sending a message is an ordinary HTTP POST that is validated, rate-limited and saved to MySQL, and only then broadcast. The gateway (Laravel Reverb, Soketi or Pusher) just holds sockets and pushes. The database is the source of truth, and the socket is a best-effort notification pipe.

Figure 1: Send over HTTP, save to MySQL, then fan out through Redis pub/sub to whichever node holds the socket.

Ordering, delivery and resync

Each conversation gets a monotonic seq number so clients can sort messages and notice gaps. A client-generated client_msg_id makes retried sends idempotent, as in Day 9.

Be precise about delivery states. "Sent" means saved on the server, "delivered" means the recipient's device acknowledged it, and "read" means they opened it. An offline user has no socket, so they get a push notification through the Day 13 service, and when they reconnect the app asks for everything after seq N.

Redis pub/sub is fire-and-forget, so if a node is down when a message is published, that live copy is lost. The client resyncs from the DB on reconnect, which makes the loss harmless.

In production, the load balancer also has to support the WebSocket upgrade.

Pause and think: Your client receives messages with seq 41 and then 43. What should it do?

Answer: Treat it as a gap and fetch from the API: GET /conversations/{id}/messages?after_seq=41. Message 42 is safe in MySQL even if its live event got lost on the way.

The design problem

Design the marketplace's buyer-merchant chat plus live order tracking on Laravel Reverb, MySQL and Redis, for around 50k concurrent connections.

Show how a message is sent and delivered across nodes, and what happens when a phone loses its connection or a WebSocket node dies. Cover read receipts, typing indicators and image messages, and decide whether order tracking needs WebSockets at all.

Try the problem before reading the solution.

Questions to think about

1. Why not just send chat messages over the socket?
2. A user's phone drops off the network for a while. How do they catch up?
3. Is SSE enough for order tracking?

Model solution

Reconnects and gaps

On every reconnect, the client calls GET /conversations/{id}/messages?after_seq=N and GET /orders/{id}, then resubscribes. It does this unconditionally, without waiting to detect a gap, because a missed message only shows up as a gap once a newer one arrives, and that might not happen for a long time. Missed live events don't matter, because the DB holds the truth. While online, a seq gap (41 followed by 43) also triggers a fetch.

Figure 2: Reconnect, fetch everything after the last seq, then resubscribe.

A WebSocket node dies

Its clients reconnect through the load balancer to another node and run the same resync, since nothing durable lived on the failed node. For deploys, drain each node, close its sockets with a "please reconnect" code, and stagger the rollout.

Read receipts and typing

The client updates last_read_seq, debounced by 1 to 2 seconds and only when the value changes, and the other side receives a read event. Typing uses Reverb client "whisper" events, which are relayed but never stored, and the UI hides the indicator after about 3 seconds.

Images

The client uploads directly to storage with a presigned URL and then sends a message carrying the attachment key. Files never go through the socket.

Capacity

50k connections across 2 or 3 Reverb nodes is reasonable once the open-file limits are raised and an event-loop extension is installed. Scale by adding nodes. Redis pub/sub handles this volume easily.

Answers to the questions

1. If you only send over the socket, a message to an offline user, or one caught on a dying node, exists nowhere. You also lose history, ordering, idempotency, and your normal auth, validation and rate limiting (Day 7). Going through HTTP and the DB first gives you all of that back, and the socket becomes a delivery hint.
2. On reconnect the client fetches everything after its last seq, then resubscribes. Nothing is lost, because every message was saved to MySQL before it was broadcast.
3. For order tracking alone, yes. SSE is one-way, runs over plain HTTP, gets through proxies more easily, and reconnects by itself with Last-Event-ID. If the app already keeps a WebSocket open for chat, though, just add an order channel on that same connection.

Interview one-liner

"The database is the source of truth. The socket is a best-effort notification layer, and clients resync by sequence number on reconnect."

Conclusion

Save first, broadcast second, and let clients catch up by sequence number, so a lost live event never costs you a message. Once that holds, the socket can stay a simple best-effort pipe.

Try it yourself

Install Laravel Reverb in a fresh project and build a two-user chat where sending is a normal POST that saves a message with a per-conversation seq and then broadcasts it.

Open two browser tabs, then stop the Reverb server (or cut the machine's network) rather than using DevTools offline mode, which may not close an already-open WebSocket. Send a few messages from one tab while Reverb is down; the POSTs still save them. Start Reverb again, and the other tab should reconnect and immediately fetch everything after its last seq.

---

This is part of my daily System Design Drills series.