Functional and Non-Functional Requirements: Latency, Throughput, Availability and Scale

"Design Instagram." Two words, and a candidate who starts drawing boxes immediately has already made a dozen decisions without noticing: that there are videos (or not), that there are stories (or not), that the system serves the whole world (or one country), that a photo must never be lost (true), that a like count must be exact at every instant (probably not). Every one of those silent decisions changes the design.

Requirements are where you make those decisions out loud. This article shows you how to ask the right questions, how to split requirements into functional and non-functional, and how to turn words like "fast" and "reliable" into numbers you can design against.

• The difference between functional and non-functional requirements, with examples.
• A reusable list of clarifying questions for any design prompt.
• What scalability, latency, throughput, availability, reliability, durability and consistency mean, in plain words and as numbers.
• Why percentiles matter more than averages, and how a single slow dependency hurts many requests.
• How to calculate the availability of a chain of components, and what redundancy buys.
• How requirements conflict, and how to state a trade-off.
• How to spend the first five minutes of an interview.

Architecture is a set of answers. Requirements are the questions. If you skip the questions, you end up answering ones nobody asked.

Two systems that sound similar can need completely different designs:

Both are "food ordering". Only the requirements tell you which one you are building.

A functional requirement describes a behaviour: something a user or another system can do, or something the system does on its own. They are usually verbs.

For our running example, the photo-sharing app Pixly from article 01:

1. A user can upload a photo with a caption.
2. A user can follow other users.
3. A user can see a home feed of recent photos from people they follow.
4. A user can like and comment on a photo.
5. The system creates thumbnails for each uploaded photo.

Just as important is what is out of scope. In an interview you have 45 minutes, so you agree with the interviewer to leave things out:

• Out of scope: video, stories, direct messages, search, ads, content moderation details.

Writing the out-of-scope list protects you. If the interviewer later asks about search, you can say "we left search out; shall I add it?" instead of looking unprepared.

These questions work for almost any prompt. You will not ask all of them every time; pick the ones that change the design.

Good questions have a reason you can state. "Is a short delay acceptable for like counts? Because if so, I can update them asynchronously and cache them aggressively" shows the interviewer why you are asking.

A non-functional requirement (also called a quality attribute) describes a property of the system as a whole: how fast, how available, how correct, how secure. These drive most architecture decisions. Two systems with the same features but different non-functional requirements look completely different.

The rule that matters most: a non-functional requirement must be measurable. "The feed should be fast" cannot be designed for or tested. "The feed loads in under 300 ms for 99% of requests" can.

Scalability is the ability to handle more load by adding resources, without redesigning the system. Load can mean more users, more requests, more data or larger items.

• Vertical scaling (scale up): a bigger machine with more CPU, memory or disk. Simple, but there is a ceiling, and it is still one machine.
• Horizontal scaling (scale out): more machines sharing the work. Much higher ceiling, but the system must be designed so work can be split.

A useful way to think about it: in a scalable system, doubling the hardware roughly doubles the capacity. In a poorly scalable one, some shared component (often a single database) becomes the limit, and adding machines elsewhere does nothing.

Measurable form: "Handle 5,000 photo uploads per second at peak, and grow to 3 times that without redesign."

These two are often confused.

• Latency is how long one request takes, from the moment it is sent to the moment the response arrives. Measured in milliseconds.
• Throughput is how many requests the system completes per unit of time. Measured in requests per second (often written RPS or QPS, queries per second).

An analogy: a highway. Latency is how long it takes one car to drive from city A to city B. Throughput is how many cars arrive at city B per hour. Adding lanes increases throughput without making any single car faster. A higher speed limit lowers latency.

The two are linked by a simple and very useful relationship called Little's Law:

requests in progress at once  =  throughput  x  average latency

If your service handles 2,000 requests per second and each takes 0.2 seconds, then on average 2,000 x 0.2 = 400 requests are in progress at any moment. If one server can handle 100 concurrent requests (for example, it has 100 worker threads), you need at least 4 servers, plus headroom. You will use this in article 07.

Notice what happens when latency rises. If a slow database makes each request take 1 second instead of 0.2, the same 2,000 RPS now means 2,000 requests in progress, five times more. Threads, connections and memory run out. Slowness turns into overload. That is why latency is not only a user-experience concern.

Suppose you measure 100 requests. Ninety-eight take about 20 ms and two take 3 seconds. The average is roughly (98 x 20 + 2 x 3,000) / 100 = 79.6 ms. That number describes nobody: almost everyone got 20 ms, and two people waited 3 seconds.

Percentiles describe the distribution instead:

• p50 (median): half of requests are faster than this. The typical experience.
• p95: 95% of requests are faster. Only 1 in 20 is slower.
• p99: 99% are faster. The "tail" that 1 in 100 requests experiences.

Google's SRE book recommends percentiles over averages for exactly this reason: averages hide the slow tail that real users feel (Service Level Objectives).

Why should you care about 1 in 100? Because one page load is often many requests. If a page needs 100 backend calls and each has a 1% chance of being slow, the chance that at least one is slow is:

1 - 0.99^100  =  1 - 0.366  =  about 63%

So a p99 problem in one service becomes a problem for most page loads. This is called tail latency amplification, and it is why mature systems set targets on p99, not on the average.

Measurable form: "Feed read: p50 under 100 ms, p99 under 300 ms, measured at the server."

Availability is the fraction of time (or of requests) for which the system works. It is usually written as a percentage with "nines".

These figures come from the availability table in Google's SRE book (Availability Table). Each extra nine allows ten times less downtime and usually costs far more than ten times as much engineering, because it rules out manual fixes: nobody can wake up, log in and fix a problem in 8.64 seconds.

You can measure availability two ways:

• Time-based: uptime divided by total time.
• Request-based: successful requests divided by total requests. This is usually more meaningful for web services, because "up but failing 30% of requests" is not really up.

Availability in series.

When a request must pass through several components, and the request fails if any of them fails, their availabilities multiply:

Load balancer 99.99%  x  App 99.9%  x  Database 99.95%  = 0.9999 x 0.999 x 0.9995  = about 0.9984, or 99.84%

The chain is less available than its weakest link. Every dependency you add to the critical path lowers the total. That is one reason to keep the synchronous path short and push non-essential work to the background (article 05).

Availability with redundancy.

When you run two copies of a component and the system works if either copy works, the probability that both fail at once is the product of their failure probabilities:

One server:  99%  available, 1% unavailableTwo servers: unavailable only if both fail = 0.01 x 0.01 = 0.0001             available = 1 - 0.0001 = 99.99%

Reliability is the system doing the right thing: returning correct results, not corrupting data, not charging twice. A system can be available but unreliable. A search page that responds instantly with wrong results is "up" but useless.

Measurable form: "No duplicate charges. Fewer than 0.1% of requests return a server error."

Durability is the promise that data, once acknowledged as saved, is not lost, even if machines, disks or a whole data centre fail. It is different from availability: a system can be temporarily unavailable (you cannot read your photo for ten minutes) while still being perfectly durable (the photo is safe and comes back).

Durability is achieved by keeping several copies in places that fail independently. For example, Amazon states that S3 Standard is "designed to provide 99.999999999% durability and 99.99% availability of objects over a given year", and stores objects across a minimum of three Availability Zones (Data protection in Amazon S3). Notice that the durability figure (eleven nines) is far higher than the availability figure (four nines). Losing data is much worse than being briefly unreachable, so durability is engineered to a much higher standard.

Measurable form: "A photo acknowledged as uploaded is never lost. Target: no data loss from any single-zone failure."

Consistency, in system design, usually means: after data changes, do all readers see the change, and how soon? (The word means something different in ACID transactions; article 12 covers that meaning.)

• Strong consistency: once a write succeeds, every later read sees it. Simple to reason about, harder to make fast and highly available across many machines.
• Eventual consistency: after a write, different readers may see old data for a short time, but all copies converge if no new writes arrive. Easier to scale and keep available.

The key insight: different parts of one product need different consistency. For Pixly:

Article 19 makes these ideas precise. For now, learn to ask "which data must be exactly right, right now?" for every design.

Who can read and change what? Is there personal data, payment data or health data? Must data stay in a particular country? Measurable forms include "all traffic encrypted in transit", "only the owner and followers can see a private account's photos", "access to payment data is logged". Article 28 covers the architecture.

• Maintainability: can a team of five change this safely every week?
• Cost: what is the budget? A design that needs 2,000 servers for a hobby project is wrong, however elegant.
• Observability: can we tell what is happening inside the system (article 25)?
• Operability: can it be deployed, scaled and recovered without heroics?

Production teams formalise non-functional requirements with three related terms, defined in Google's SRE book (Service Level Objectives):

You do not need SLAs in an interview, but stating your non-functional requirements as SLO-style targets ("99.9% of uploads succeed, p99 under 2 seconds") is an easy way to sound precise. Article 25 returns to SLOs and alerting.

You will learn to produce the traffic numbers in articles 06 and 07.

Non-functional requirements pull against each other. Recognising the tension and saying which side you chose is a large part of what interviewers grade.

A good way to state a trade-off has three parts: what you chose, what it costs, and why the cost is acceptable here.

"I'll update like counts asynchronously and cache them for a few seconds. The cost is that counts can lag by a few seconds. That's acceptable because likes are not financial data, and it removes most of the write load from the hot path."

Here is what a complete requirements section looks like for Pixly. In an interview this would be spoken and written as short notes, in about five minutes.

Functional (in scope)

1. Upload a photo (up to 10 MB) with a caption.
2. Follow and unfollow users.
3. View a home feed of photos from followed users, newest first.
4. Like a photo; see like counts.

Out of scope: video, comments, search, direct messages, moderation, ads.

Scale (assumptions, to be estimated in article 06): 50 million registered users, 10 million daily active, mostly in one region.

Non-functional

Notice how much design this already implies. "Thumbnails can follow later" suggests a queue and workers. "Never lost, even if one data centre fails" suggests object storage that keeps copies in several zones. "Followers within 10 seconds" means the feed does not need to be strongly consistent, which opens the door to precomputing feeds. You have not drawn a single box, and the shape of the design is already visible.

A company asks you to design an internal tool where 300 HR staff run monthly payroll for 20,000 employees.

Functional: upload salary changes, run the payroll calculation, approve, export bank payment files, view history.

Non-functional:

The right design here is modest: a well-run relational database with backups and point-in-time recovery, a single application with strong authorisation and audit logging, and a background job for the payroll run. Pixly's design would be wrong for this, and this design would be wrong for Pixly. Same method, different requirements, different architecture.

• Designing before asking. The interviewer expects questions. Silence followed by a diagram is a red flag.
• Unmeasurable targets. "Highly available" means nothing until it has a number.
• Asking for five nines by default. Each nine costs a lot. Ask what the business actually needs.
• One consistency level for everything. Choose per feature.
• Using averages for latency. Talk about p95 or p99.
• Forgetting durability. Being briefly down is recoverable; losing data often is not.
• Too many features. Three to five core features, clearly agreed, beat ten vague ones.
• Not writing down the out-of-scope list. It saves you later in the interview.

1. Restate the problem in one sentence.
2. Ask five to eight clarifying questions, each with a reason.
3. Write three to five functional requirements and an out-of-scope list.
4. Write the non-functional requirements as numbers, and mark the one or two that will drive the design (for example "read-heavy, p99 under 300 ms" or "no double booking").
5. Confirm with the interviewer: "Does this match what you have in mind?"

Then move to estimation. Article 29 turns this into a timed checklist.

12, 15, 14, 13, 16, 18, 15, 14, 13, 17, 19, 15, 14, 16, 900, 15, 13, 14, 16, 1200

• Next: How a Web Request Travels: DNS, TCP, TLS and HTTP for System Design follows one request through the network, so you can see where latency comes from.
• Numbers: articles 06 and 07 turn "10 million daily users" into requests per second, storage and bandwidth.
• Going deeper: availability and redundancy return in article 24, consistency in article 19, and SLOs in article 25.