Design a Chat System Like a Messaging App

A chat app feels simple: type, send, it appears on your friend's phone. Behind that, the system must keep tens of millions of connections open, find which of hundreds of servers holds the recipient's connection, keep messages in order, never lose one, never show one twice, deliver to people who are offline, and keep every device a person owns in sync. And it must do it in a fraction of a second.

Chat is a favourite interview question because it forces you to deal with stateful connections, ordering, delivery guarantees and partitioned storage all at once. This case study works through it with the course method.

• How to scope a messaging product and estimate connections, messages and storage.
• Why chat uses persistent connections, and how to scale stateful gateways.
• How a session registry routes a message to the right server.
• Per-conversation sequence numbers for ordering, acknowledgements and deduplication.
• Offline delivery, push notifications and sync on reconnect, across several devices.
• Group chat fan-out, presence and typing indicators.
• How the design fails, and how it is secured and observed.

Build a messaging service with one-to-one chats and group chats (up to 500 members), delivered in real time to online users and reliably to offline users, with message history available on all of a user's devices. Users can send text and media (photos and short videos) and see delivery and read receipts.

Functional requirements

1. Send and receive messages in one-to-one and group conversations, in order.
2. Deliver in real time to online devices; deliver later (with a push alert) to offline devices.
3. Sync history to every device; fetch older messages by scrolling.
4. Delivery and read receipts; presence and typing indicators.
5. Send media (via object storage).

Out of scope: voice and video calls, broadcast channels with huge audiences, end-to-end encryption design, search across messages.

Non-functional requirements

These figures continue the worked example from articles 06 and 07 (assumptions, not any real product's numbers):

100 million DAU; 40 messages sent per user per dayMessages: 4 billion/day -> about 46,000/s average, about 92,000/s at peak (factor 2)Deliveries (fan-out): 80% one-to-one (1 recipient), 20% to groups (average 20 members, 19 others)  deliveries per message = 0.8 x 1 + 0.2 x 19 = 4.6; x devices online per recipient (say 1.3)  -> about 92,000 x 4.6 x 1.3 = about 550,000 device deliveries/s at peakConcurrent connections at peak: assume 30% of DAU online, 1.3 devices each  -> about 39 million concurrent connectionsStorage: average message about 200 bytes raw, about 400 bytes stored with overhead  -> 4 billion x 400 B = 1.6 TB/day, about 580 TB/year (before replication)Media: stored in object storage, referenced by key (not in the message store)

Conclusions:

• Connections: about 39 million long-lived connections. If one gateway server holds about 100,000 connections comfortably (an assumption to confirm with load tests, since it depends on memory, TLS and traffic per connection), we need about 400 gateway servers, plus headroom and zone redundancy.
• Writes: about 92,000 messages per second at peak into a store growing by more than half a petabyte a year: a partitioned, write-optimised store (article 17).
• Fan-out: deliveries are about 6 times the message rate, so routing must be efficient.

Real-time traffic flows over a WebSocket connection (article 05), which gives both sides a persistent, full-duplex channel (RFC 6455). Messages on it are small JSON (or binary) frames:

Client -> server: SEND   { client_msg_id: "c-7f3a", conversation_id: "cv_88", body: "On my way", media_key: null }Server -> client: ACK    { client_msg_id: "c-7f3a", conversation_id: "cv_88", seq: 1042, server_ts: "..." }Server -> client: MSG    { conversation_id: "cv_88", seq: 1042, sender_id: 42, body: "On my way", server_ts: "..." }Client -> server: DELIVERED { conversation_id: "cv_88", up_to_seq: 1042 }Client -> server: READ      { conversation_id: "cv_88", up_to_seq: 1042 }Client -> server: TYPING    { conversation_id: "cv_88" }          (ephemeral, not stored)

Ordinary HTTP APIs handle everything that is not real-time:

GET  /v1/conversations?cursor=...                         -> my conversations with last message and unread countGET  /v1/conversations/{id}/messages?before_seq=1000&limit=50   -> history, scrolling backGET  /v1/conversations/{id}/messages?after_seq=1042       -> sync after reconnectPOST /v1/conversations                                    -> create a groupPOST /v1/media/uploads                                    -> presigned upload URL for a photo or video

Two identifiers do a lot of work:

• client_msg_id, generated by the sending device, makes SEND idempotent: if the device retries after a lost acknowledgement, the server recognises it and returns the same seq.
• seq, assigned by the server per conversation, defines the order and lets devices detect gaps.

-- Wide-column style (article 17): partition by conversation and time bucket, cluster by sequence.CREATE TABLE messages (    conversation_id  text,    bucket           text,          -- for example '2026-10'; bounds partition size    seq              bigint,    client_msg_id    text,    sender_id        bigint,    body             text,    media_key        text,    server_ts        timestamp,    PRIMARY KEY ((conversation_id, bucket), seq)) WITH CLUSTERING ORDER BY (seq DESC);

conversations         (conversation_id PK, type one_to_one | group, created_at, last_seq)conversation_members  (conversation_id, user_id, role, joined_at)       PK (conversation_id, user_id)user_conversations    (user_id, last_activity_at, conversation_id, unread_count)                      partition user_id, sorted by last activity -> "my chat list"device_cursors        (user_id, device_id, conversation_id, delivered_seq, read_seq)dedupe                (conversation_id, client_msg_id) -> seq            (short TTL, for retries)

• The hottest access pattern, "latest 50 messages of a conversation", is one range read on one partition.
• user_conversations is a denormalised read model (article 27) for the chat list, updated on each message.
• device_cursors records how far each device has received and read, which powers sync and receipts.
• Partition keys: conversation_id (plus bucket) for messages; user_id for chat lists and cursors.

Phone A --WebSocket--> Chat server --> DatabasePhone B --WebSocket--/      (if B is connected to the same server, push MSG to B)

One server holds all connections, saves each message and forwards it to the recipient's connection. That works for thousands of users. It cannot hold 39 million connections, and with many servers, the recipient's connection is usually on a different server from the sender's.

1. Connections: tens of millions of long-lived connections need hundreds of stateful servers.
2. Routing: a message arrives at gateway 17, but the recipient's devices are connected to gateways 203 and 331.
3. Ordering: two group members send at the same moment through different servers; all devices must show the same order.
4. Reliability: connections drop constantly on mobile networks; acknowledgements get lost; senders retry.
5. Offline users: most recipients are not connected at any moment.
6. Storage writes: about 92,000 messages per second at peak.

                    +-------------------------------+Devices --WS-->     | Gateway servers (stateful)    |  hold connections, authenticate,                    | ~400+ across zones            |  forward frames; no business logic                    +-------------------------------+                         |   ^                 heartbeats: register device -> gateway                         v   |                         |                    +--------------------+      +-----------------------+                    | Message service    |----->| Session registry      |  user/device -> gateway                    | (stateless)        |      | (Redis, TTL heartbeats)|                    +--------------------+      +-----------------------+                         |            +------------+-------------+-------------------+            v                          v                   v     Message store             Delivery fan-out       Notification system     (partitioned by           (routes to gateways    (push for offline devices,      conversation)             via internal pub/sub)  article 33)

• Gateways are deliberately thin: they terminate TLS and WebSockets, authenticate the device at connect time, keep connections alive with heartbeats, and pass frames to and from the message service. Keeping logic out of them means they rarely need redeploying, and each redeploy disconnects users.
• The message service is stateless and does the real work.
• The session registry maps each online device to its gateway. Gateways register devices on connect and refresh entries with a short TTL (say 60 seconds, renewed every 20), so entries for crashed gateways disappear on their own.

1. A's device sends SEND{client_msg_id, conversation_id, body} to its gateway.2. Gateway forwards to the message service.3. Message service:   a. checks A is a member of the conversation (authorisation);   b. checks dedupe(conversation_id, client_msg_id): if present, re-sends the ACK with the stored seq and stops;   c. assigns seq = next sequence number for the conversation (see below);   d. writes the message to the message store (durable, replicated);   e. records dedupe(conversation_id, client_msg_id) -> seq;   f. sends ACK{client_msg_id, seq} back to A's device;   g. publishes "deliver seq to the conversation's members" for fan-out.4. Fan-out: for each member device, look up the registry:   - online: route MSG to that device's gateway;   - offline: request a push notification (article 33) and rely on sync later.5. Recipient devices reply DELIVERED / READ with up_to_seq; cursors are updated and receipts relayed.

The sender's device shows a "sent" tick only after the ACK, which means the message is durably stored. Until then it shows "sending" and retries with the same client_msg_id.

The sequence must be assigned in one place per conversation, so concurrent senders get distinct, increasing numbers. Options:

• An atomic counter per conversation in a fast store (for example an atomic increment in Redis, or UPDATE conversations SET last_seq = last_seq + 1 ... RETURNING last_seq). Simple, but the counter store must be durable enough that sequence numbers are never reused after a failover (a reused number would collide with a stored message, which the primary key would reject, prompting a retry with the next number).
• Single-writer partitions: route all messages for a conversation to one owner (for example by hashing conversation_id to a partition of a log such as Kafka, which preserves order within a partition (Introduction to Kafka)), and let that owner assign the sequence. Ordering and assignment come from the same place.

Either way, ordering is per conversation, not global, which keeps it cheap and parallel (article 18: never use wall clocks from different servers to order messages).

The message service must reach the gateways holding recipients' connections:

• Direct: look up the gateway in the registry and call it (an internal RPC). Simple and fast; the service needs addresses of all gateways.
• Pub/sub: each gateway subscribes to a channel named after itself (or after the users it holds), and the service publishes deliveries to those channels. Redis Pub/Sub is one lightweight option (Redis Pub/Sub); note that its messages are fire-and-forget (not stored if nobody is listening), which is acceptable here because the durable copy is already in the message store and devices sync by sequence.

Losing a real-time delivery is not data loss: the device will notice a gap in sequence numbers, or fetch on reconnect.

• If no device of a recipient is online, the fan-out asks the notification system to send a push ("Asha: On my way"), respecting preferences (article 33). Push providers deliver to phones even when the app is closed (Firebase Cloud Messaging).
• When a device connects, it sends its delivered_seq per conversation (or one "sync token"), and the service returns everything newer, newest conversations first. History older than what the device holds is fetched only when the user scrolls (cursor pagination by seq).
• Gap detection: if a device receives seq 1045 but last had 1042, it fetches 1043 and 1044 before displaying in order.

Each device has its own cursor in device_cursors. A message is delivered to every device of each member, including the sender's other devices (so the conversation looks the same on the sender's laptop). Read state syncs across devices: reading on the phone clears the unread badge on the laptop.

A group message is stored once (in the conversation's partition) and delivered to each member's devices. With up to 500 members, fan-out is at most a few thousand device deliveries per message, handled asynchronously by fan-out workers in batches. For very large groups, receipts become expensive (500 "read" receipts per message); systems often aggregate them ("read by 12") or disable per-person receipts above a size.

Presence ("online", "last seen 5 minutes ago") changes constantly. Pushing every change to every contact is costly: a user with 300 contacts going online and offline 50 times a day generates 15,000 updates. Keep it cheap:

• Presence derives from the session registry (a device is online if its registry entry is fresh).
• Fetch presence on demand when a user opens a chat, and subscribe only for the chats currently open on screen.
• Typing indicators are ephemeral frames relayed to online members of an open conversation, never stored, rate-limited (at most one every few seconds per user).
• Treat both as lossy and eventually consistent; nothing bad happens if one is missed.

Photos and videos are uploaded directly to object storage with a presigned URL (article 10; presigned URLs); the message carries only the object key and a thumbnail. Recipients download through a CDN with short-lived signed URLs, so only conversation members can fetch media.

Messages go to a wide-column store partitioned by (conversation_id, bucket), replicated three ways across zones, with writes acknowledged by a quorum (article 19); Apache Cassandra, for example, documents this Dynamo-style partitioning and tunable consistency (Cassandra: Dynamo). The relational database keeps accounts, conversations and memberships, where relationships and constraints matter. Old buckets can move to cheaper storage tiers.

• Within a conversation: a total order defined by seq, identical on every device.
• Across conversations: no ordering guarantee is needed.
• Delivery: at-least-once to devices; devices deduplicate by (conversation_id, seq), so users never see duplicates.
• Read-your-writes: the sender's device shows its own message immediately (optimistically), confirmed by the ACK; its other devices receive it like any member.
• Receipts, unread counts, presence: eventually consistent.
• Durability: a message is acknowledged only after it is written to a quorum of replicas.

• Transport security: TLS on all connections.
• Authentication at WebSocket connect (a short-lived token), with periodic re-validation; revoked sessions are disconnected.
• Authorisation on every send and every history read: the user must be a current member of the conversation (object-level checks, article 28). Removing a member must stop future deliveries to them.
• Abuse: rate limits on sends per user and per new account; spam detection on content and patterns; limits on group creation and invites; report and block features.
• Media: scanned for malware; signed, short-lived download URLs.
• Privacy: presence and last-seen visibility controlled by user settings; minimal metadata in logs.
• End-to-end encryption (extension): encrypting messages so that only participants' devices hold keys changes the design significantly: the server stores ciphertext, cannot search or moderate content, and multi-device and group key management become the hard problems. Mention it as a deliberate scope decision.

"We support one-to-one and group chats for 100 million daily users: about 92,000 messages a second at peak, roughly 550,000 device deliveries a second, and 39 million concurrent connections.

Devices hold WebSocket connections to thin, stateful gateways, about 400 of them across zones. Gateways register each device in a Redis session registry with heartbeat TTLs. A stateless message service handles each send: it checks membership, deduplicates by the client's message id, assigns a per-conversation sequence number in one place, writes the message to a wide-column store partitioned by conversation and month with quorum replication, and only then acknowledges the sender.

Fan-out looks up each member's devices in the registry and routes the message to their gateways; offline devices get a push via the notification platform and sync from their last delivered sequence when they reconnect. Sequence numbers give the same order on every device, let devices detect gaps, and make retries harmless, so delivery is at-least-once with client-side deduplication. Presence and typing are lossy and fetched on demand. If a gateway dies, clients reconnect with jittered backoff and resync, so nothing acknowledged is lost."

By memory: 32 GB / 30 KB = 32 x 10^9 / 30 x 10^3 = about 1.07 million connectionsGateways for 39 million: about 37 (by memory alone)

100M users x 40 changes/day x 200 contacts = 800 billion updates/day/ 86,400 = about 9.3 million updates per second

• Next: Design a News Feed: Fan-Out, Ranking and the Celebrity Problem, a different fan-out problem where precomputation and reading at request time compete.
• Related: the ride-hailing design (article 41) uses the same real-time connection machinery for live locations.