Estimating Storage, Bandwidth, Cache Size and Server Count

Article 06 turned users into requests per second. That tells you how often things happen, but not how big they are. A system that handles 70 uploads per second sounds easy until you notice each upload is 2 MB and they never get deleted. A feed that serves 3,500 requests per second sounds manageable until you notice each one pulls ten images.

This article finishes the estimation toolkit: storage, bandwidth, cache memory and server count. Each section ends with the design decision the number drives, because that is the only reason to compute it.

• How to estimate the size of one record and one blob, with overheads.
• Storage per day and per year, with retention and replication.
• Ingress and egress bandwidth, in bytes and bits per second, average and peak.
• How to size a cache from the working set, and what the hit ratio does to database load.
• How to estimate server count from per-server capacity or from Little's Law, with headroom and zone failure.
• How to check whether one database can take the write load.
• A complete worked example for Pixly, and a calculator.

List the fields and add up their sizes. For a Pixly photo's metadata row:

Databases add overhead: row headers, indexes, free space in pages. Assume the stored size is roughly double the raw size unless you know better. Here that gives about 600 bytes; round up to 1 KB to keep the arithmetic easy and leave room for new fields.

Use the average size of variable fields, not the maximum. A caption may be up to 2,200 characters, but the average is far smaller. Using maximums overestimates storage by an order of magnitude.

Small structured records (metadata, likes, follows) belong in a database. Large unstructured files (photos, videos, documents) belong in object storage (article 10), with the database holding only a pointer. Estimate them separately, because they have different costs, different growth and different stores.

For a Pixly photo, assume an average original of 2 MB after the app compresses it, plus three thumbnails totalling 300 KB: about 2.3 MB of blob data per upload.

Storage per day   = items per day x bytes per itemStorage per year  = storage per day x 365Total footprint   = storage per year x years retained x number of copies

Retention matters: if data is kept forever, storage only grows. If logs are kept for 30 days, storage is capped at 30 days' worth.

Copies matter too. A database with one primary and two replicas stores three copies. Managed object storage handles redundancy internally (Amazon S3, for example, stores objects across at least three Availability Zones for most storage classes), so you normally count only the logical size when estimating object storage, but you would count each database replica.

Design conclusions:

• Photos are petabytes per year. They must go to object storage; no database or file server on application machines is appropriate.
• Metadata is only a few terabytes per year. A single well-provisioned relational database can hold years of it; partitioning is a later problem, not a day-one problem.
• The likes table grows fastest among database tables, almost three times faster than photo metadata. It is the first candidate for partitioning or for a store built for high write volume (articles 16 and 17).

Bandwidth is bytes per second moving into the system (ingress) and out of it (egress). Compute it from the daily volume, then apply the peak factor, then convert to bits.

Average bytes per second = bytes per day / 86,400Peak bytes per second    = average x peak factorBits per second          = bytes per second x 8

4.6 TB per day / 86,400 s = about 53 MB/s averagex 8 = about 0.43 Gbps average; x 3 peak = about 1.3 Gbps

Assume each feed view loads 10 images of about 100 KB each (feed-sized, not originals): 1 MB per feed view.

100M feed views x 1 MB = 100 TB per day100 TB / 86,400 s = about 1.16 GB/s, x 8 = about 9.3 Gbps averagePeak (x 3) = about 28 Gbps

Egress is about 20 times ingress. That is typical of media products: content is created once and viewed many times.

A CDN (article 10) serves repeated requests for the same files from edge locations, so they never reach your servers. If 90% of image requests are hits at the CDN (an assumption that depends heavily on how concentrated viewing is), your origin only serves the remaining 10%:

Origin egress at peak = 28 Gbps x (1 - 0.9) = about 2.8 Gbps

Design conclusion: without a CDN, Pixly's own servers and network would need to push about 28 Gbps at peak, and users far away would wait on long round trips. With one, the origin sees a tenth of that. For any media-heavy design, the bandwidth estimate is what justifies the CDN.

A cache keeps copies of frequently read data in memory (article 09). Memory is far more expensive per byte than disk, so you cannot usually cache everything. You cache the hot working set: the part of the data that receives most reads.

Many access patterns are skewed: a small fraction of items gets most of the reads. A common planning heuristic is the "80/20" assumption, that about 20% of items receive about 80% of reads. It is an assumption, not a law; check it against real access logs when you have them.

Example: Pixly caches photo metadata for photos from the last 7 days, since the feed mostly shows recent photos.

2M new photos per day x 7 days x 1 KB = 14 GB

Fourteen gigabytes fits comfortably in the memory of one cache node. With a replica for availability, that is two nodes, which is a very cheap way to take most feed reads off the database.

A bigger example: suppose Pixly precomputes each active user's feed as a list of the latest 500 photo ids (article 35 discusses whether to).

10M DAU x 500 ids x 8 bytes = 40 GB (plus data-structure overhead; assume x2 = 80 GB)

Still feasible, but now spread over a few nodes. If each node offers about 50 GB of usable memory (an assumption about the instance size you choose), that is 2 nodes, or 4 with replicas.

The hit ratio is the fraction of reads served from the cache. What reaches the database is the rest:

Database reads = total reads x (1 - hit ratio)

(35,000 lookups per second is the 3,500 feed reads per second at peak, each needing metadata for 10 photos.)

Going from 90% to 99% cuts database load by ten times. Small improvements in hit ratio are worth a lot. The reverse is also true, and dangerous: if the cache is lost and the hit ratio falls to zero, the database suddenly faces ten or a hundred times its usual load. Articles 09 and 23 deal with that failure.

There are two sound ways to estimate how many application servers you need. Both depend on knowing what one server can do, which you measure with a load test. In an interview, state it as an assumption.

Assume load testing shows one API server handles 500 feed requests per second while keeping p99 latency within target.

Servers for peak = 3,500 / 500 = 7

From article 02: requests in progress = throughput x latency. Assume an average latency of 100 ms and that one server handles 50 concurrent requests well.

Concurrent requests at peak = 3,500 x 0.1 s = 350Servers = 350 / 50 = 7

Both methods give 7, which is a good sign that the assumptions are consistent.

Seven servers running at 100% at peak is a fragile system. Add:

1. Utilisation headroom. Aim for at most about 60% utilisation at peak, so latency stays low and small spikes are absorbed: 7 / 0.6 = about 12 servers.
2. Zone failure. Spread servers across 3 availability zones (separate data centres in one region). If one zone fails, the other two must carry the whole peak. Each zone must therefore hold half the requirement: 12 / 2 = 6 per zone, 18 servers in total.

Raw need:            7With 60% target:    12Survive 1 of 3 zones lost: 18 (6 per zone)

The jump from 7 to 18 surprises people, but it is the real cost of availability. Autoscaling can reduce the off-peak count, but you still need enough capacity to survive a zone failure at the peak.

Writes are often the first database limit, because they cannot be spread over replicas. Pixly's peak writes are about 1,700 likes plus 70 uploads per second, about 1,800 writes per second.

Whether one primary handles that depends on the hardware, the schema, the indexes and the size of each write, so it must be measured. If a load test shows the primary can sustain, say, 5,000 writes per second of this shape at the target latency, you are at about 36% of capacity: comfortable today, with room to grow about 2.5 times before you need batching, a different store for likes, or partitioning. If the test shows 2,000, you are already near the limit, and the design must change now.

This is the pattern: the estimate tells you which question to ask a load test.

Each application server keeps a pool of database connections. 18 servers x 20 connections = 360 connections. Some databases handle very large numbers of connections poorly (PostgreSQL, for example, uses a process per connection), so systems often put a connection pooler between applications and the database. Article 12 covers this.

This Java program reproduces the storage and bandwidth figures above. As before, it models the arithmetic; a spreadsheet does the same job.

// File: CapacityPlan.javaimport java.util.Locale;public class CapacityPlan {    static final double DAY = 86_400;    static final double KB = 1e3, MB = 1e6, GB = 1e9, TB = 1e12, PB = 1e15;    static String bytes(double b) {        if (b >= PB) return String.format(Locale.US, "%.1f PB", b / PB);        if (b >= TB) return String.format(Locale.US, "%.1f TB", b / TB);        if (b >= GB) return String.format(Locale.US, "%.1f GB", b / GB);        if (b >= MB) return String.format(Locale.US, "%.1f MB", b / MB);        return String.format(Locale.US, "%.1f KB", b / KB);    }    static String gbps(double bytesPerSecond) {        return String.format(Locale.US, "%.1f Gbps", bytesPerSecond * 8 / 1e9);    }    public static void main(String[] args) {        double uploadsPerDay = 2_000_000;        double photoBytes = 2 * MB + 300 * KB;   // original plus three thumbnails        double metaBytes = 1 * KB;        double feedViewsPerDay = 100_000_000;        double bytesPerFeedView = 10 * 100 * KB; // ten images of about 100 KB        double peak = 3;        double photoPerDay = uploadsPerDay * photoBytes;        System.out.println("Photo storage per day:  " + bytes(photoPerDay));        System.out.println("Photo storage per year: " + bytes(photoPerDay * 365));        double metaPerYear = uploadsPerDay * metaBytes * 365;        System.out.println("Metadata per year, 3 copies: " + bytes(metaPerYear * 3));        double ingress = photoPerDay / DAY;        System.out.println("Upload ingress avg / peak: " + gbps(ingress) + " / " + gbps(ingress * peak));        double egress = feedViewsPerDay * bytesPerFeedView / DAY;        System.out.println("Image egress avg / peak:   " + gbps(egress) + " / " + gbps(egress * peak));        double cdnHitRatio = 0.9;        System.out.println("Origin egress at peak with 90% CDN hits: " + gbps(egress * peak * (1 - cdnHitRatio)));    }}

Photo storage per day:  4.6 TBPhoto storage per year: 1.7 PBMetadata per year, 3 copies: 2.2 TBUpload ingress avg / peak: 0.4 Gbps / 1.3 GbpsImage egress avg / peak:   9.3 Gbps / 27.8 GbpsOrigin egress at peak with 90% CDN hits: 2.8 Gbps

Seven numbers, and the shape of the architecture is already clear: stateless API servers behind a load balancer, a cache, a relational database with replicas, object storage behind a CDN, and queues for thumbnails and fan-out. The next module builds each of those blocks.

• Disk fills up: storage was estimated without retention or growth. Monitor free space and alert well before it runs out.
• Bandwidth saturates: a viral item or a CDN misconfiguration (a sudden fall in hit ratio) sends far more traffic to the origin than estimated.
• Cache loss overloads the database: the hit ratio drops from 99% to 0% and database reads rise a hundredfold.
• Zone loss overloads the survivors: capacity was planned for the peak but not for the peak minus a zone.
• Connections exhausted: many servers each with large pools exceed the database's connection limit after a scale-out.

• Using maximum field sizes instead of averages.
• Putting large files in the database "because it is simpler".
• Forgetting replicas when estimating database storage.
• Quoting bandwidth in bytes when links are sized in bits.
• Sizing the cache for the whole dataset instead of the hot set.
• Computing servers from the average instead of the peak, then forgetting headroom and zone loss.
• Treating per-server capacity as a known fact instead of something to measure.

After traffic, spend two or three minutes on:

1. Storage: bytes per item, items per day, per year, with copies. Separate blobs from metadata.
2. Bandwidth: egress at peak, especially for media. Decide on a CDN.
3. Cache: the hot set in GB, and whether it fits on a few nodes.
4. Servers and database: only if asked, or if writes look close to a single-machine limit.

Then say the conclusions in one sentence each, and move on to the API.

Raw size per message = 200 bytes; stored about 400 bytesPer day = 4 x 10^9 x 400 bytes = 1.6 x 10^12 bytes = 1.6 TBPer year = 1.6 TB x 365 = about 584 TB, about 0.6 PBTwo years = about 1.2 PBThree copies = about 3.5 PB

Bytes per second per listener = 160,000 bits / 8 = 20,000 bytes = 20 KB/sPer user per day = 3,600 s x 20 KB = 72 MBPer day = 20M x 72 MB = 1.44 x 10^15 bytes = 1.44 PBAverage egress = 1.44 x 10^15 / 86,400 = about 16.7 GB/s = about 133 GbpsPeak (x2) = about 267 GbpsOrigin at peak with 95% hits = 267 x 0.05 = about 13 Gbps

Hot products = 2% of 5M = 100,000Memory = 100,000 x 4 KB = 400 MB (assume x2 for cache overhead = 800 MB)

Raw: 12,000 / 800 = 15 serversAt 65% utilisation: 15 / 0.65 = about 23.1, round up to 24Survive 1 of 3 zones: the remaining 2 zones must hold 24, so 12 per zoneTotal: 36 servers (12 per zone)

• Next: Load Balancers: How They Work, Algorithms, Health Checks and Stateless Services spreads those 18 servers' worth of traffic, and explains why the servers must be stateless.
• The blocks these numbers justified: caching (article 09), CDNs and object storage (article 10), queues (article 11), and the database articles (12 to 17).