Skip to content
SparkTime, home
System Design From Zero to Interview-Ready
Lessons
1 · System Design Foundations
2 · APIs and Communication
3 · Capacity Estimation
4 · Core Building Blocks
5 · Databases
6 · Distributed Systems
7 · Reliability and Operations
8 · Architecture and Security
9 · Interview Method and Foundation Case Studies
10 · Real-World Case Studies
11 · Large-Scale Case Studies
12 · Interview Readiness
Lesson 3 of 45

Lesson 3 · System Design Foundations

How a Web Request Travels: DNS, TCP, TLS and HTTP for System Design

Follow one request from a phone to a database and back: DNS lookup, TCP and TLS handshakes, HTTP, the edge, the load balancer, the application and the data stores. Learn where the time goes, why round trips and distance dominate latency, and where requests fail.

On this page

Key takeaways

  • A request crosses DNS, a TCP connection, a TLS handshake and an HTTP exchange before your code even runs, and each step can cost a network round trip.
  • Round trips and physical distance dominate latency for users far from your servers, so reuse connections, avoid chatty APIs and serve content close to users.
  • DNS caching makes lookups fast but also slows down failover, because old answers live until their TTL expires.
  • HTTP is stateless: any state between requests must travel with the request or live in a shared store, which is what makes horizontal scaling possible.
  • Every hop is a place where a request can fail or wait, so each needs a timeout and a plan.

You tap a link to a photo on Pixly. Half a second later it appears. In that half second your phone found an address for a name, opened a connection across several networks, agreed encryption keys with a server it had never met, sent a request, and waited while that request passed through a load balancer, an application server, a cache and maybe a database, before the answer came all the way back.

You do not need to be a network engineer to design systems. But you do need a working model of this path, because almost every design decision about latency, scaling and failure is really a decision about one of these hops. This article gives you that model.

What you will learn

#
  • The steps a request goes through, from the client to the database and back.
  • What DNS does, how caching and TTLs work, and how DNS is used for routing and failover.
  • What TCP and TLS handshakes cost, and why round trips matter so much.
  • What HTTP requests and responses contain, and how HTTP/1.1, HTTP/2 and HTTP/3 differ at a high level.
  • How to build a simple latency budget and see where time goes.
  • Rough latency numbers worth remembering.
  • Where along the path requests fail, and what that means for design.

The journey in one picture

#
Code
 Phone / browser   |  1. DNS lookup: "photos.pixly.example" -> IP address   |  2. TCP connection to that address (handshake)   |  3. TLS handshake (encryption keys, certificate check)   |  4. HTTP request: GET /photos/123   v CDN / edge server ------ static file cached here? return it (fast path)   |  (cache miss or dynamic request)   v Load balancer ---------- picks a healthy application server   v Application server ----- authenticates, runs business logic   |        \   |         \--> Cache (is the answer already in memory?)   v Database --------------- reads or writes the data   |   v Response travels back the same way: app -> load balancer -> edge -> client

Steps 1 to 3 happen before your application code sees anything. They are often skipped in interviews, yet for users far from your servers they can take longer than everything else combined.

Step 1: DNS turns a name into an address

#

Computers route traffic to IP addresses (for example 203.0.113.10), but people use domain names (photos.pixly.example). The Domain Name System (DNS) is the distributed directory that maps one to the other. Its design is described in RFC 1034 (Domain Names: Concepts and Facilities).

How a lookup works

#
  1. Your device asks a recursive resolver, usually run by your internet provider or a public DNS service: "What is the address of photos.pixly.example?"
  2. If the resolver does not have the answer cached, it asks a root server, which points it to the servers for .example.
  3. Those top-level domain (TLD) servers point it to the authoritative servers for pixly.example, which are run by Pixly or its DNS provider.
  4. The authoritative server returns the address. The resolver caches it and returns it to your device, which also caches it.
Code
Device -> Recursive resolver -> Root -> TLD (.example) -> Authoritative (pixly.example)   <------------------- answer, cached for TTL seconds ---------------------

TTL: the caching dial

#

Every DNS answer comes with a TTL (time to live), in seconds. Resolvers and devices may reuse the answer until it expires. This makes almost all lookups fast, because most are answered from a nearby cache.

TTL is a trade-off you will meet again:

TTL

Advantage

Disadvantage

Long (hours)

Fewer lookups, lower latency, less load on DNS servers

If you need to move traffic (an outage, a migration), clients keep using the old address until their cache expires

Short (tens of seconds)

Traffic can be moved quickly

More lookups; and some clients and resolvers do not honour very short TTLs perfectly

DNS as a routing tool

#

Managed DNS services do more than answer a fixed address. Amazon Route 53, for example, is described in its documentation as offering domain registration, DNS routing and health checking (What is Amazon Route 53?). Similar features exist in other providers' DNS services. Common uses in system design:

  • Geographic or latency-based routing: users in India get the address of servers in an Indian region; users in Europe get European servers.
  • Weighted routing: send 5% of traffic to a new deployment to test it.
  • Failover: health checks watch your servers; if the primary region stops responding, DNS starts returning the address of a standby.

Keep the TTL trade-off in mind: DNS failover is only as fast as clients' caches expire. That is why many systems use DNS for coarse routing between regions and use load balancers for fast failover within a region (article 08).

Step 2: TCP opens a reliable connection

#

Most web traffic today runs over TCP (Transmission Control Protocol), which gives the two ends a reliable, ordered stream of bytes over an unreliable network. It handles lost packets (by resending), reordering (by sequence numbers) and pacing (so a fast sender does not overwhelm a slow network). HTTP/3 is the exception: it runs over QUIC on top of UDP, covered below.

Before any data flows, TCP performs a three-way handshake: the client sends a SYN, the server replies SYN-ACK, the client sends ACK. That costs one round trip before the client can send its request.

A round trip time (RTT) is how long a message takes to go to the other end and back. It is set mostly by physical distance (light in fibre is fast but not instant) and by the number of network hops. Rough orders of magnitude:

Path

Typical RTT (order of magnitude)

Two servers in the same data centre

Well under a millisecond

User to a server in the same metro area

A few to tens of milliseconds

User in India to a server in Western Europe or the US

Roughly 150 to 300 milliseconds

These are rough figures for reasoning, not measurements of any particular network. Real values depend on routes, mobile networks and congestion. The point is the ratio: crossing continents costs hundreds of times more than staying inside a data centre.

Step 3: TLS makes the connection secure

#

HTTPS is HTTP inside TLS (Transport Layer Security). TLS gives three guarantees:

  • Confidentiality: nobody on the path can read the traffic.
  • Integrity: nobody can change it without detection.
  • Authentication: the client can check, using the server's certificate, that it is really talking to photos.pixly.example and not an impostor.

To get there, client and server perform a TLS handshake to agree on keys. In TLS 1.3, the current version, a full handshake takes one round trip, and a client resuming an earlier session can even send data in its first flight, called 0-RTT data, though the specification warns that such data can be replayed (RFC 8446, TLS 1.3). Older TLS 1.2 handshakes took two round trips.

Where TLS ends: termination

#

Decrypting traffic costs CPU and requires the private key. Many systems terminate TLS at the edge: the CDN or load balancer decrypts the request and forwards it to application servers over the internal network. This centralises certificate management and offloads work from the app servers.

The trade-off: traffic between the load balancer and the app servers is then plain text unless you re-encrypt it. Systems handling sensitive data often re-encrypt internal traffic too, so that a compromised internal machine cannot read everything. Article 28 returns to this.

Step 4: HTTP carries the request and response

#

Now, finally, the actual request. HTTP is a request-response protocol whose meaning is defined in RFC 9110 (HTTP Semantics). A request has:

  • a method (GET, POST, PUT, PATCH, DELETE and others) saying what kind of action it is;
  • a target, such as /photos/123;
  • headers: key-value metadata such as Authorization, Accept, Cache-Control;
  • an optional body, for example the JSON of a new comment.

A response has a status code, headers and an optional body:

Code
GET /photos/123 HTTP/1.1Host: api.pixly.exampleAuthorization: Bearer (token)Accept: application/jsonHTTP/1.1 200 OKContent-Type: application/jsonCache-Control: private, max-age=60{"id": 123, "owner": "asha", "caption": "Sunset", "likes": 1203}

Status codes fall into families: 2xx success, 3xx redirection, 4xx a problem with the request (the client should change something), 5xx a problem on the server. Article 04 covers which code to use when.

HTTP is stateless

#

Each HTTP request stands alone. The server is not required to remember anything about earlier requests from the same client. If the application needs continuity (who is logged in, what is in a cart), that state must either travel with each request (a cookie or token) or be looked up in a shared store (a database or cache).

This is not a limitation to work around; it is what makes large systems possible. If no application server holds private memory of a user, then any server can handle any request, and a load balancer can spread requests freely. Article 08 builds on this.

HTTP versions, briefly

#

Version

Runs over

What changed (high level)

HTTP/1.1

TCP

Persistent connections (keep-alive) so one connection serves many requests, but one request at a time per connection

HTTP/2

TCP

Many requests and responses interleaved over one connection (multiplexing), header compression (RFC 9113)

HTTP/3

QUIC over UDP

Same ideas as HTTP/2, but on QUIC, which combines the transport and encryption handshakes and avoids one lost packet stalling every stream (RFC 9114)

For system design you rarely need more than this. The practical lessons: reuse connections instead of opening a new one per request, and know that newer protocols reduce handshake and loss-related delays, which matters most on mobile networks and long distances.

Steps 5 to 7: through the edge, the load balancer and the application

#

Once the request leaves the client:

  1. CDN or edge server. If the request is for a static file (an image, a script) that the edge has cached, it answers immediately from a location close to the user. Otherwise it forwards the request towards your servers, often over a faster, better-connected network. Article 10 covers CDNs.
  2. Load balancer. It picks one healthy application server and forwards the request. It may also terminate TLS, add headers such as the original client address, and enforce limits. Article 08 covers load balancers.
  3. Application server. It authenticates the caller, validates the input and runs the business logic. To answer, it may check a cache (article 09), query a database (articles 12 to 17), call other internal services, or put a message on a queue for background work (article 11).
  4. The response returns along the same path. The browser may then fetch dozens more resources (images, scripts, API calls) for one page, each with its own path through this system.

Inside your data centre, each internal hop costs well under a millisecond of network time. But they add up, and every hop is a place where a request can wait or fail.

Worked example: a latency budget

#

A latency budget splits a target ("the photo page loads in under 500 ms") into the time each step may take. Let us build one for a first visit, with a fresh connection.

Assumptions (illustrative, not measurements):

  • The user is in Bengaluru.
  • Option A: Pixly's servers are in a region in Mumbai, about 20 ms RTT away.
  • Option B: Pixly's servers are in the eastern US, about 250 ms RTT away.
  • The DNS answer is already cached on the device (common for repeat visits; a cold lookup would add more).
  • TLS 1.3 with a full handshake (1 RTT).
  • Server-side work (load balancer, app, cache, database) takes 50 ms in total.

Step

Round trips

Option A (20 ms RTT)

Option B (250 ms RTT)

DNS (cached)

0

0 ms

0 ms

TCP handshake

1

20 ms

250 ms

TLS 1.3 handshake

1

20 ms

250 ms

HTTP request and response

1

20 ms

250 ms

Server-side work

n/a

50 ms

50 ms

Total

3

110 ms

800 ms

Look at Option B. The server-side work is 50 ms, the same as in Option A, yet the user waits 800 ms. Distance multiplied by round trips dominates. No amount of database tuning fixes that. What does fix it:

  • Serve from closer: a region near the users, or a CDN edge for static content.
  • Reuse connections: on the second request over the same connection, TCP and TLS are already done, so Option B drops to 250 + 50 = 300 ms.
  • Avoid sequential round trips: if the page needs three API calls one after another, each costs a full RTT. Combine them into one call, or make them in parallel.
  • Use protocols that save handshakes, such as TLS 1.3 and HTTP/3, especially for mobile users.

This is why "chatty" APIs (many small calls to build one screen) are a design smell for mobile clients far from the server.

Latency numbers worth knowing

#

You do not need exact figures, but you should know the orders of magnitude, because they tell you which operations are worth worrying about. The table below gives rough values in the spirit of the widely shared "latency numbers every programmer should know" list; an interactive version by Colin Scott shows how such estimates change by year and makes clear they are extrapolations, not measurements (Latency numbers).

Operation

Rough order of magnitude

Read from main memory

About 100 nanoseconds

Random read from an SSD

Tens of microseconds

Round trip inside one data centre

About half a millisecond

Read 1 MB sequentially from an SSD

Around a millisecond

Disk seek on a spinning hard drive

About 10 milliseconds

Round trip between continents

About 150 milliseconds or more

What to take from this:

  • Memory is far faster than disk, and disk far faster than crossing the world. That is the whole case for caches (keep hot data in memory) and CDNs (keep content near users).
  • One cross-continent round trip costs as much as hundreds of thousands of memory reads. Optimising code while ignoring network round trips is usually optimising the wrong thing.
  • Inside a data centre, a network call is cheap but not free. A request that makes 50 sequential internal calls at half a millisecond each spends 25 ms just on network time.

What this means for system design

#

Observation

Design consequence

Round trips are expensive over distance

Reuse connections, batch calls, design APIs that return what a screen needs in one call

Distance dominates for global users

CDNs for static content; regional deployments for dynamic content when needed

DNS answers are cached for their TTL

DNS failover is slow; use load balancers for fast failover inside a region

HTTP is stateless

Keep application servers stateless; put state in shared stores; this enables horizontal scaling

TLS can terminate at the edge

Simpler certificate management; decide whether internal traffic must be re-encrypted

Every hop can fail or stall

Every outbound call needs a timeout; critical paths should have as few hops as possible

Where requests fail

#

When you design, walk the path and ask "what if this step fails?"

Step

Example failure

What the user sees

Typical mitigation

DNS

Misconfigured record; DNS provider outage

"Server not found"

Managed DNS with redundancy; careful change process; sensible TTLs

TCP

Server overloaded, cannot accept connections

Connection timeouts

Load balancing, autoscaling, connection limits

TLS

Expired or wrong certificate

Browser security error, requests refused

Automated certificate renewal and expiry alerts

Load balancer

No healthy targets; misrouted traffic

502 or 503 errors

Health checks, multiple zones, gradual config rollouts

Application

Bug, crash, slow dependency

500 errors or very slow responses

Redundant instances, timeouts, circuit breakers (article 23)

Database

Overload, failover in progress

Slow or failed writes

Replicas, failover, caching, backpressure

Two habits follow from this table and will appear in every case study:

  1. Every network call gets a timeout. Without one, a slow dependency holds threads and connections forever, and slowness turns into an outage (remember Little's Law from article 02).
  2. Every component in the critical path is a possible point of failure. Fewer hops in the synchronous path means fewer ways to fail.

Trade-offs to remember

#

Decision

Option A

Option B

DNS TTL

Long: fast, cheap, slow to change

Short: quick failover, more lookups

TLS termination

At the edge: simple, offloads CPU

End to end: internal traffic also encrypted, more certificate work

Connection strategy

New connection per request: simple, slow

Pooled or kept-alive connections: fast, must be managed and limited

Where to serve from

One region: simpler, cheaper

Several regions or edges: lower latency for distant users, harder data consistency

Common mistakes

#
  • Ignoring the network in latency estimates. Server time is often the smaller part for distant users.
  • Assuming DNS failover is instant. Cached answers keep old addresses alive until their TTL ends.
  • Opening a new connection for every call between services, paying handshakes each time.
  • Designing chatty APIs for mobile clients, so each screen costs several sequential round trips.
  • Keeping user state in one server's memory, which breaks as soon as there are two servers.
  • Forgetting certificate expiry, a common and entirely avoidable cause of outages.
  • No timeouts on outbound calls.

In an interview

#

You rarely need to explain TCP in detail. What earns credit is using the model:

  • "Users are in India, so I'll deploy in an Indian region and put static images behind a CDN to cut round-trip time."
  • "The mobile home screen makes one aggregated call instead of five, because each sequential call costs a full round trip on a mobile network."
  • "Application servers are stateless; sessions live in a shared store, so the load balancer can send any request anywhere."
  • "TLS terminates at the load balancer; internal traffic is re-encrypted because we handle payment data."

Exercises

#

Exercise 1

Sequential calls

#

A mobile screen needs data from three API endpoints, called one after another because each needs the previous result. The RTT to the server is 200 ms, each request takes 30 ms on the server, and the connection is new (TCP and TLS 1.3, one round trip each; DNS cached).

  1. How long until the screen has all its data?
  2. How long if the connection is already open?
  3. How long if a new endpoint returns all three results in one call over an open connection, with 60 ms of server time?

Exercise 2: DNS failover

#

Pixly uses DNS failover between two regions, with a TTL of 1 hour on its main record. The primary region fails at 10:00. Health checks detect it at 10:01 and DNS starts returning the standby region's address. Roughly when will all users be on the standby? What would you change, and what does the change cost?

Exercise 3: Diagnose by symptom

#

For each symptom, name the most likely step on the path and one thing you would check:

  1. Every user suddenly gets a browser warning that the connection is not private.
  2. Users in Europe see 2-second page loads; users in India see 200 ms; server metrics are identical for both.
  3. The load balancer returns 503 Service Unavailable for every request, but the application servers' processes are running.
  4. A new domain name works on some phones but not others for about an hour after it was set up.

Exercise 4: Stateless or not?

#

An application server keeps each logged-in user's shopping cart in a map in its own memory. The site grows from one server to four behind a load balancer. What goes wrong, and give two ways to fix it?

Show solutionHide solution
  1. New connection: TCP (200 ms) + TLS (200 ms) = 400 ms of setup. Then three sequential requests, each one RTT plus server time: 3 x (200 + 30) = 690 ms. Total: about 1,090 ms.
  2. Open connection: no setup. 3 x 230 = 690 ms.
  3. One aggregated call over an open connection: 200 + 60 = 260 ms.

The server work barely changed (90 ms versus 60 ms). What changed was the number of round trips. For distant or mobile users, reducing sequential round trips is usually the biggest latency win available.

Exercise 2

Show solutionHide solution

Resolvers and devices that looked up the record just before 10:00 may keep the old address for up to an hour, so some users keep trying the failed region until about 11:00. Users whose caches expire earlier move sooner. In practice a few clients cache even longer than the TTL.

Changes:

  • Lower the TTL on records used for failover, for example to 60 seconds. Cost: many more DNS queries (more cost, slightly more lookup latency on cache misses), and failover still is not instant.
  • Put a global load balancer or anycast address in front of both regions, so the address users hold does not change and routing changes happen behind it. Cost: another component and usually higher price.
  • Make clients retry against an alternative endpoint if the first fails. Cost: client complexity.

The broader lesson: DNS is good for steering traffic, but slow for emergency failover. Plan your recovery time with DNS caching in mind (article 24).

Exercise 3

Show solutionHide solution
  1. TLS. Most likely an expired or misconfigured certificate. Check its expiry date and the automated renewal job.
  2. Network distance. Servers are probably only in or near India. Check where European traffic is served from. Fix with a CDN for static content and possibly a European region for dynamic content.
  3. Load balancer health checks. The processes run, but the load balancer considers no target healthy. Check the health-check path, port and expected response, and whether a dependency used by the health check (such as the database) is failing.
  4. DNS propagation and caching. Some resolvers cached a "does not exist" answer or an older record. Check the TTLs, including how long negative answers are cached, and wait for caches to expire.

Exercise 4

Show solutionHide solution

With four servers, the load balancer may send a user's next request to a different server, which has no record of their cart. Items "disappear" and reappear randomly. A server restart also loses every cart it held.

Fixes:

  • Move the cart to a shared store, such as a database or a cache like Redis, keyed by user id or session id. Any server can then read it. This is the usual answer and keeps servers stateless.
  • Sticky sessions: configure the load balancer to send each user to the same server. This works short-term but spreads load unevenly, and carts are still lost when that server dies. Article 08 discusses why stateless servers are usually better.

A third option for small carts is to keep the cart on the client (in a cookie or local storage). That avoids server state, but limits size and requires care with security and validation.

Where to go next

#

Sources & further reading

  1. rfc-editor.org
  2. rfc-editor.org
  3. rfc-editor.org
  4. rfc-editor.org
  5. rfc-editor.org
  6. colin-scott.github.io
  7. docs.aws.amazon.com
  • Guide · Technology

    How to Learn Programming When AI Can Already Write Code

  • Guide · Technology

    Start Here: Your Roadmap from Zero to Job-Ready DSA

  • Guide · Career

    A LinkedIn Profile That Gets Recruiter Messages