Synchronous vs Asynchronous Communication: Queues, Polling, WebSockets and Server-Sent Events

A user uploads a photo to Pixly. Behind that one tap, the system must store the file, save its metadata, create three thumbnail sizes, scan the image for policy violations, and tell the uploader's followers. If the upload request does all of that before replying, the user stares at a spinner for several seconds, and if the follower-notification step fails, the whole upload fails with it, even though the photo itself was saved perfectly well.

The fix is to ask a simple question about every piece of work: does the caller need the result right now? If yes, do it while they wait. If no, accept the request, reply quickly, and do the rest in the background. This article is about making that decision well, and about the ways to deliver results and live updates back to users.

• What synchronous and asynchronous communication mean, and what each costs.
• A decision guide for when to wait and when to defer work.
• Patterns for returning the result of background work: status polling, webhooks and push.
• Short polling, long polling, Server-Sent Events, WebSockets and mobile push notifications, compared.
• The failure modes of each approach and how to handle them.
• Two worked examples: Pixly's upload flow and a live-score page with millions of viewers.

In synchronous communication the caller sends a request and waits for the response before continuing. An ordinary HTTP API call is synchronous from the caller's point of view.

Client ----request----> Service A ----request----> Service B       <---response----           <---response----         (client waits the whole time)

Why it is attractive:

• Simple to write and reason about: call, get answer, continue.
• Errors come back immediately, to the code that can handle them.
• The caller knows the outcome when the call returns.

What it costs:

• Latency adds up. If A calls B, which calls C, the user waits for all three in sequence.
• Availability multiplies. As article 02 showed, every synchronous dependency in the path lowers the availability of the whole chain.
• Coupling in time. Both sides must be up and fast at the same moment. A slow B makes A slow; a down B makes A fail.
• Load passes straight through. A traffic spike at the front hits every service in the chain at the same instant.

In asynchronous communication the caller hands off work and does not wait for it to finish. Usually the work is written to a message queue (article 11), and the caller gets a quick acknowledgement that the work was accepted. Workers pick up messages and do the work later, often within milliseconds or seconds.

Client --request--> API server --put message--> Queue --> Worker (does the slow work)       <--202 / 201--         (replies at once)

Why it is attractive:

• Fast responses. The user gets an answer as soon as the essential part is done.
• Spikes are absorbed. The queue acts as a buffer: if 10,000 uploads arrive in a minute, workers process them at their own steady pace instead of the system collapsing. Microsoft describes this as the queue-based load leveling pattern (Queue-Based Load Leveling).
• Failures are isolated. If the notification provider is down, uploads still succeed; notification messages wait in the queue and are retried.
• Independent scaling. You can add workers without touching the API servers.

What it costs:

• Delay. The work is done "soon", not "now". Users may see "processing" states.
• The result must be delivered somehow (the next section).
• Duplicates and ordering. Most queues deliver a message at least once, so a worker may see it twice; messages may not arrive in order (article 20).
• More moving parts. A queue to run and monitor, workers to deploy, a backlog that can grow silently.
• Harder debugging. One user action now spans several processes and times; you need tracing (article 25).

Ask these questions about each step of a request:

1. Does the caller need the result to continue? Login must tell you whether you are logged in. Sending a welcome email does not need to finish before you see your dashboard.
2. Is the step slow? Anything that can take more than a few hundred milliseconds (image or video processing, report generation, calls to slow third parties) is a candidate for the background.
3. Is the step unreliable? Calls to external providers (email, SMS, payment confirmations) fail and need retries; retrying inside a user's request makes them wait and multiplies load.
4. Can it be retried safely later? Background work must be idempotent or deduplicated, because it will sometimes run twice.
5. Is the load spiky? Bursty work benefits from a buffer.

Most real requests are a mix: a synchronous core that does the minimum the user needs, and asynchronous follow-up work. The art is keeping the synchronous core small.

If work happens later, how does anyone learn the outcome? Three common patterns.

The API accepts the job, returns 202 Accepted and a URL to check progress (article 04 introduced this):

POST /v1/exports                 -> 202 Accepted, Location: /v1/exports/exp_31GET  /v1/exports/exp_31          -> {"status": "running"}GET  /v1/exports/exp_31          -> {"status": "done", "download_url": "..."}

Simple, works with any client, and the status resource is also useful for support staff. The client must poll, which is the subject of the next section.

When the caller is another system, your service can call its URL when the work completes. Payment providers use this heavily: the provider sends an HTTP request to the merchant's server when a payment succeeds or fails. Stripe's documentation, for example, describes webhooks as the way to receive events from its platform and asks receivers to verify signatures and handle duplicate events (Webhooks).

If you send webhooks, plan for receivers being down: retry with backoff, sign each payload so receivers can verify it came from you, and include an event id. If you receive webhooks, verify the signature, respond quickly (do the processing asynchronously), and make the handler idempotent, because the same event may arrive more than once.

For people using an app, the most pleasant option is to push the result: a notification, or a live update on screen. The next section covers how.

HTTP was designed for the client to ask and the server to answer. Real-time features need the server to tell the client something it did not ask for at that moment. There are five common ways.

The client asks every few seconds: "anything new?"

Client: GET /notifications?since=...   (every 5 s)Server: 200 [] ... 200 [] ... 200 [one item] ...

• Pros: trivial to build; works everywhere; each request is an ordinary stateless HTTP call, so it scales with normal load balancers and can be cached by a CDN when every client asks the same question.
• Cons: wasteful when updates are rare (most responses are empty); updates arrive late by up to one polling interval.

The waste is easy to estimate. One million connected users polling every 5 seconds generate 1,000,000 / 5 = 200,000 requests per second, almost all returning nothing.

The client asks, and the server holds the request open until there is something to send or a timeout (say 30 seconds) passes. Then the client immediately asks again.

• Pros: near-real-time delivery without new protocols; far fewer empty responses than short polling.
• Cons: the server holds many open requests, which needs an efficient, non-blocking server; timeouts in proxies and load balancers must be configured; each message still costs a new request.

The client opens one HTTP connection, and the server keeps it open and streams events down it as text. The browser's EventSource API handles parsing and automatically reconnects, sending the id of the last event it received so the server can resume. SSE is specified in the HTML standard (Server-sent events).

• Pros: simple, one-way server push over plain HTTP; built-in reconnection and resume; works with HTTP/2.
• Cons: one direction only (client-to-server messages use normal requests); text-based; each client holds an open connection.

A WebSocket starts as an HTTP request and then upgrades to a persistent, full-duplex connection: both sides can send messages at any time with very little overhead per message (RFC 6455, The WebSocket Protocol).

• Pros: true two-way, low-latency messaging; ideal for chat, collaborative editing, multiplayer games, live trading screens.
• Cons: stateful connections. Each connection lives on one specific server, which changes how you scale and deploy (below). You also build your own message format, acknowledgements and reconnection logic.

When a phone app is in the background or closed, it has no open connection. Mobile platforms provide their own push services, such as Firebase Cloud Messaging for Android (it can also deliver to iOS through Apple's service) and the Apple Push Notification service for iOS (Firebase Cloud Messaging). Your backend sends a message to the platform service, which delivers it to the device.

• Pros: reaches users when the app is not running; battery-friendly because the operating system manages one shared connection.
• Cons: delivery is best-effort and not instant; payload size is limited; you depend on a third party; users can disable notifications. Treat push as a nudge ("you have a new message"), and fetch the real data when the app opens.

SSE and WebSockets change the scaling picture, because a connection is pinned to one server for its whole life.

• Capacity is measured in concurrent connections, not only requests per second. A server holding 50,000 idle connections uses memory for each, even if no messages flow.
• Messages must find the right server. If user A's connection is on server 3 and user B sends A a message through server 7, server 7 must route it to server 3. Common solutions are a connection registry (user id to server, kept in a fast shared store) plus internal messaging, or a pub/sub system that every connection server subscribes to. Article 34 designs this for chat.
• Load balancers must allow long connections: idle timeouts long enough, and protocol upgrade support for WebSockets.
• Deployments disconnect users. Restarting a server drops all its connections. Drain connections gradually and let clients reconnect.
• Reconnection storms. If a server with 100,000 connections crashes, 100,000 clients reconnect at once. Clients must reconnect with exponential backoff and random jitter (article 20) so they spread out.
• Missed messages. While disconnected, a client misses events. Give events increasing ids, and let clients resume from the last id they saw, or fetch what they missed through a normal API.

Let us apply the decision guide to the photo upload from the introduction.

REQUEST PATH (synchronous, user waits)  App --> Load balancer --> API server --> Database (metadata, status = processing)                                 |                                 +--> Queue: "photo.uploaded" (photo id)  App <-- 201 Created {photoId, status: "processing"}ASYNCHRONOUS PATH (user does not wait)  Queue --> Thumbnail workers ---> Object storage (thumbnails)        --> Moderation workers --> Database (status = visible or rejected)        --> Feed workers --------> Followers' feeds        --> Notification workers -> Push provider

The uploader sees their photo immediately (marked as processing), so the "read your own writes" requirement from article 02 is met. Followers see it seconds later, which the requirements allowed. If the push provider is down, notifications wait in the queue and the upload is unaffected.

The sharp edge: the API writes to the database and puts a message on the queue. If the server crashes between the two, the photo exists but no thumbnails are ever made. Article 21 solves this "dual write" problem with the transactional outbox pattern. For now, notice that splitting work into sync and async parts creates a new consistency question at the boundary.

A sports site shows a live cricket score. During a big match, 2 million people have the page open. The score changes every 30 seconds or so; commentary lines appear every few seconds. Viewers only read; they never send data. How should updates reach them?

Option A: WebSockets. Real-time, but 2 million persistent two-way connections need a large fleet of connection servers, a fan-out system to push each update to all of them, and careful handling of reconnection storms. The two-way capability is wasted, because viewers never send anything.

Option B: SSE. One-way, simpler than WebSockets, with built-in resume. Still 2 million long-lived connections to hold and fan out to.

Option C: short polling of a cached endpoint. Every client requests GET /matches/123/live every 5 seconds. That is 2,000,000 / 5 = 400,000 requests per second, which sounds alarming. But every viewer asks the same question. Put the endpoint behind a CDN with a 2-second cache lifetime. Each CDN location then fetches from the origin at most once every 2 seconds, no matter how many viewers it serves. With, say, a few hundred edge locations, the origin sees a few hundred requests per second, not 400,000. The CDN, which is built for exactly this, absorbs the rest.

For a score that changes every 30 seconds, a few seconds of delay is acceptable, and Option C is cheapest and most robust. If the product required sub-second updates (betting odds, for instance), SSE through infrastructure built for streaming would be justified. The requirement decides, not the technology's popularity. This is exactly the kind of reasoning interviewers want to hear.

• Making everything async "for scale". Async adds real complexity. Use it where the decision guide says so.
• Doing slow third-party calls inside the user's request, so the provider's outage becomes yours.
• Async work that is not idempotent, so a redelivered message sends two emails or charges twice.
• No visibility into the background: nobody notices that the queue has been growing for six hours.
• Choosing WebSockets for one-way updates that SSE or cached polling would serve more cheaply.
• Clients that reconnect instantly and in sync after an outage, turning a blip into a storm.
• Forgetting the user experience of delay: show "processing" states, and let users see their own changes immediately.

• For each functional requirement, say which parts are synchronous and which are background work, and why.
• If you introduce a queue, say what it buys (latency, spike absorption, isolation) and what it costs (duplicates, delay).
• For any real-time feature, name the mechanism and justify it with the update frequency, direction and number of clients.
• Mention reconnection with backoff and resume from the last event for any long-lived connection.

• Next: Back-of-the-Envelope Estimation: From Users to Peak QPS gives you the numbers to judge whether a queue, a cache or another server is needed.
• Queues in depth: article 11 explains producers, consumers, acknowledgements, retries and dead-letter queues.
• Duplicates and retries: article 20 makes background work safe to run twice.
• Real-time at scale: the chat (34) and ride-hailing (41) case studies use these mechanisms under heavy load.