Design a URL Shortener (and Pastebin): A Complete Walkthrough

"Design a URL shortener" is often the first system design question people meet, and it is easy to underestimate. The functional core fits in one sentence: turn a long URL into a short code, and redirect anyone who visits the code. Yet a good answer touches estimation, key generation, caching, consistency, failure handling and abuse, and the same reasoning applies to any read-heavy lookup service.

This is the first full case study of the course. Every case study from here on uses the same fourteen sections, which follow the seven phases from article 29. Read it once straight through, then try the problem yourself with a 45-minute timer before rereading.

• How to scope a deceptively simple problem.
• How to estimate a read-heavy system and what the numbers imply.
• Three ways to generate short codes, and how to handle collisions.
• Why the choice between 301 and 302 redirects matters.
• How to scale the redirect path with caching, replicas and, eventually, sharding.
• How to add analytics and expiry without slowing redirects.
• How the design fails and how it is secured.
• How the same design becomes Pastebin.

Users submit a long URL such as https://www.example.com/articles/2026/10/how-to-plan-a-budget-trip-to-kerala?ref=newsletter and receive a short link such as https://pix.ly/a8Kq2Zx. Anyone who opens the short link is redirected to the original URL. The service is public: anonymous users and signed-in users can create links, and links are shared widely on social media and in messages.

Questions worth asking (and assumed answers):

Functional requirements

1. Create a short link for a long URL, optionally with a custom alias and an expiry time.
2. Redirect a short link to its long URL.
3. Let owners delete their links.
4. Show owners click counts per link (minutes of delay acceptable).

Out of scope: link editing, detailed analytics dashboards, QR codes, team workspaces.

Non-functional requirements

The requirement that drives the design: redirects are an extremely read-heavy, latency-sensitive key lookup that must be very highly available.

Writes (new links)

100 million per month / (30 x 86,400 s) = about 40 per second on averagePeak factor 3: about 120 per second

Reads (redirects)

100 redirects per link -> 10 billion redirects per month10 billion / 2.6 million s = about 3,900 per second on averagePeak factor 3: about 12,000 per secondRead:write ratio: 100 to 1

Storage

Per link: code (7 bytes), long URL (average about 200 bytes, maximum 2 KB),owner, timestamps, expiry, flags: about 500 bytes with overheadsPer year: 1.2 billion links x 500 bytes = 600 GBFive years: 6 billion links, about 3 TB (before replicas)

Cache

Access is skewed: recent and viral links get most clicks. Assume the hottest 20 million links (about the last few weeks of popular links) serve most redirects: 20 million x 500 bytes = 10 GB. That fits comfortably in memory on a small replicated cache cluster.

Bandwidth

A redirect response is tiny (headers and a short body, under 1 KB): 12,000 per second x 1 KB = 12 MB per second, about 0.1 Gbps. Bandwidth is not a concern.

Conclusions: writes are modest (one database primary is plenty); storage grows to a few terabytes over years (one database can hold it, partitioning is a later option); reads are the hard part, at 12,000 per second with a tight latency target, which calls for caching.

POST /v1/linksIdempotency-Key: 3c1f...                (so a retried create does not make two links){ "long_url": "https://www.example.com/...", "custom_alias": "kerala-trip", "expires_at": "2027-01-01T00:00:00Z" }201 CreatedLocation: /v1/links/kerala-trip{ "code": "kerala-trip", "short_url": "https://pix.ly/kerala-trip", "expires_at": "2027-01-01T00:00:00Z" }Errors: 400 malformed URL; 409 alias already taken; 422 URL not allowed (blocked domain); 429 rate limited

GET /{code}302 FoundLocation: https://www.example.com/...Errors: 404 unknown code; 410 Gone for an expired or deleted link

DELETE /v1/links/{code}          -> 204 (owner only)GET    /v1/links/{code}/stats    -> 200 {"clicks": 18234, "as_of": "..."} (owner only)

HTTP defines 301 Moved Permanently and 308 Permanent Redirect, which clients and caches may remember, and 302 Found and 307 Temporary Redirect, which they should not treat as permanent (RFC 9110).

Because we need click counts and the ability to delete and expire links, choose 302. (A shortener without analytics or expiry could prefer 301 to reduce load.) This is a good example of a small decision with consequences worth stating in an interview.

The only hot access pattern is "find the long URL by code": a point lookup by key.

CREATE TABLE links (    code        VARCHAR(32) PRIMARY KEY,       -- random code or custom alias    long_url    VARCHAR(2048) NOT NULL,    owner_id    BIGINT,                         -- null for anonymous links    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),    expires_at  TIMESTAMPTZ,    deleted_at  TIMESTAMPTZ,    is_custom   BOOLEAN NOT NULL DEFAULT false);CREATE INDEX links_by_owner ON links (owner_id, created_at DESC) WHERE owner_id IS NOT NULL;CREATE INDEX links_expiring ON links (expires_at) WHERE expires_at IS NOT NULL AND deleted_at IS NULL;CREATE TABLE link_click_counts (    code        VARCHAR(32) PRIMARY KEY REFERENCES links(code),    clicks      BIGINT NOT NULL DEFAULT 0,    updated_at  TIMESTAMPTZ NOT NULL);

• The primary key on code makes redirects a single index lookup and enforces uniqueness for both random codes and custom aliases.
• links_by_owner serves "my links" pages; partial indexes keep the expiry job cheap.
• Click counts are separate, updated in batches by the analytics pipeline, so redirects never write to links.
• Partition key if we ever shard: code. Every redirect includes it, and random codes spread evenly.

Relational or key-value? The access pattern is pure key-value, so a key-value store (for example DynamoDB or Cassandra) would fit well. A relational database also fits: a few terabytes over five years, about 120 writes per second at peak, and point reads that are mostly served by the cache. Either is defensible. Choose the one the team operates confidently, and say that the access pattern makes a later migration to a key-value store straightforward.

               create (POST)                            redirect (GET /code)Client ---------------------------> Load balancer <------------------------- Browser                                          |                                    App servers (stateless)                                          |                                     PostgreSQL (primary)

Create flow: validate the URL; generate a code (section 7); insert the row (the primary key rejects duplicates); return 201.

Redirect flow: look up code; if it is missing, deleted or expired, return 404 or 410; otherwise return 302 with the Location.

This works for a small number of users. At 12,000 redirects per second, something breaks first.

1. Database reads. 12,000 point lookups per second at peak, with a 50 ms p99 target, all hitting one primary. Even if the database could keep up, it leaves no headroom, and any slowdown shows up directly in redirect latency.
2. Code generation. How do we create unique, short, unguessable codes, possibly on many app servers at once?
3. Hot links. A viral link can receive a large share of all traffic.
4. Analytics. Counting clicks naively (an UPDATE per redirect) would add 12,000 writes per second to the database.

How long must codes be? Using the 62 characters 0-9, a-z and A-Z, 6 characters give 62^6 = about 57 billion codes and 7 characters give 62^7 = about 3.5 trillion. Five years of links (6 billion) fits in either, but random codes collide more often as the space fills, so 7 characters gives comfortable room.

Three approaches:

This Java program shows base-62 encoding and decoding of counter values, the size of the 7-character space, the collision probability for a random code once 6 billion links exist, and some random codes (with a fixed seed so the output is repeatable; production code would use a secure random source).

// File: ShortCodes.javaimport java.util.Locale;import java.util.Random;public class ShortCodes {    static final String ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ";    static final int BASE = ALPHABET.length();   // 62    // Counter-based codes: encode a unique number in base 62.    static String encode(long n) {        if (n == 0) {            return "0";        }        StringBuilder sb = new StringBuilder();        while (n > 0) {            sb.append(ALPHABET.charAt((int) (n % BASE)));            n /= BASE;        }        return sb.reverse().toString();    }    static long decode(String code) {        long n = 0;        for (char c : code.toCharArray()) {            n = n * BASE + ALPHABET.indexOf(c);        }        return n;    }    // Random codes: pick each character at random (in production, from a secure random source).    static String randomCode(Random random, int length) {        StringBuilder sb = new StringBuilder();        for (int i = 0; i < length; i++) {            sb.append(ALPHABET.charAt(random.nextInt(BASE)));        }        return sb.toString();    }    public static void main(String[] args) {        for (long n : new long[] {1, 61, 62, 125_000_000_000L}) {            String code = encode(n);            System.out.println(n + " -> " + code + " -> " + decode(code));        }        long space7 = (long) Math.pow(BASE, 7);        System.out.printf(Locale.US, "7-character space: %,d codes%n", space7);        System.out.println("largest 7-character code: " + encode(space7 - 1));        long used = 6_000_000_000L;               // 5 years of links in the worked example        System.out.printf(Locale.US, "chance a new random code collides after %,d links: %.2f%%%n",                used, 100.0 * used / space7);        Random random = new Random(7);            // fixed seed only so this output is repeatable        System.out.println("random codes: " + randomCode(random, 7) + " " + randomCode(random, 7) + " " + randomCode(random, 7));    }}

1 -> 1 -> 161 -> Z -> 6162 -> 10 -> 62125000000000 -> 2crtgcg -> 1250000000007-character space: 3,521,614,606,208 codeslargest 7-character code: ZZZZZZZchance a new random code collides after 6,000,000,000 links: 0.17%random codes: A0FQQ6c HISUmE3 jC7k3hA

Even after five years, a new random code collides with an existing one less than 0.2% of the time, so on average about one create in 600 needs a single retry, and two consecutive collisions are very rare. That retry is cheap and correct because the database's primary key is the referee: two servers that happen to generate the same code at the same instant cannot both insert it.

Decision: random 7-character codes, inserted with the primary key enforcing uniqueness, retrying up to a few times on conflict. Custom aliases use the same table and the same constraint; a conflict there returns 409 to the user instead of retrying. (If short, sequential-looking codes were a requirement, a counter with ranges allocated to each server, scrambled by a reversible permutation to avoid enumeration, would be the alternative.)

Add a distributed cache (for example Redis) in front of the database: cache-aside for reads, as described in Microsoft's Cache-Aside pattern (Cache-Aside pattern):

• Redirect: look up link:{code} in the cache; on a hit, redirect; on a miss, read the database, store the result with a TTL of about a day (with jitter), redirect.
• Create: after inserting the row, write it into the cache too, so the very first redirect (often the creator testing the link) is a hit and never depends on replica lag.
• Delete: mark the row deleted, then delete the cache entry.
• Unknown codes: cache "not found" for a short time (negative caching), so bots probing random codes do not reach the database.

With the 10 GB hot set cached and a hit ratio of, say, 95%, the database sees about 600 reads per second at peak instead of 12,000.

• Cache misses go to read replicas, not the primary, which handles only about 120 writes per second.
• Each app server keeps a tiny in-process cache (a few seconds' TTL) for the hottest codes, so a viral link does not overload one cache node (the hot-key problem from article 09).

Redirect responses for popular links can be cached at a CDN for a short time, which cuts latency for distant users and load on the origin. A CDN such as CloudFront routes each request to a nearby edge location (What is Amazon CloudFront?). The cost: clicks served at the edge are not seen by our servers, so counting must come from CDN logs, and deleting a link takes effect only after the edge TTL. Keep the edge TTL short (seconds to a minute) if used at all.

Never update a counter in the database on each redirect. Instead, after sending the redirect, the app server emits a click event (code, timestamp, coarse country and referrer) to a queue or stream, without waiting for an acknowledgement on the request path. Aggregation workers consume events, sum clicks per code per minute, and update link_click_counts in batches. If the analytics pipeline is down, redirects continue; events wait in the stream (article 11).

• On read: a redirect for an expired link returns 410 Gone, even if the row still exists. (The cached entry stores expires_at, so the cache checks it too.)
• Cleanup: a background job uses the links_expiring index to find expired links in batches, marks them, and removes their cache entries.
• Never reuse codes, even after expiry: an old printed link must not start pointing somewhere new.

At 3 TB over five years and 120 writes per second, one primary with replicas is enough for a long time. If growth is ten times faster, partition links by hash of code across shards (article 16). Every redirect includes the code, so each lookup touches one shard, and random codes spread load evenly. "My links" queries by owner would then need a secondary index partitioned by owner, or the owner's links could be kept in a separate table keyed by owner.

REQUEST PATH (redirect)Browser -> (optional CDN, short TTL) -> Load balancer -> App servers              App server: in-process hot cache -> Redis cache -> read replicas (on miss)              App server returns 302 immediatelyREQUEST PATH (create)Client -> Load balancer -> App servers -> URL safety checks -> Primary DB (unique code) -> write-through to RedisASYNC PATHApp servers --click events--> Stream --> Aggregation workers --> link_click_counts (batched)Scheduler --> Expiry job --> marks expired links, deletes cache entries

The key reliability idea: the redirect path depends on as little as possible (app server, cache, replicas). Creation, analytics and expiry can degrade without breaking redirects.

• New links: strongly visible to the creator immediately, thanks to write-through to the cache (and primary reads for very young codes on a miss).
• Deleted links: stop working after the cache entry is deleted; if edge caching is used, after its TTL as well. State this window.
• Click counts: eventually consistent, minutes behind; acceptable per requirements.
• Code uniqueness: strongly enforced by the database primary key, never by application checks.

The real-world risk of a public shortener is abuse: attackers hide phishing and malware links behind a trusted-looking short domain.

• Check URLs at creation against blocklists and reputation services, and re-scan popular links periodically; block known-bad domains (422).
• Rate-limit creation per account, API key and IP; stricter for anonymous users; CAPTCHA or account requirement above a threshold (article 31).
• Takedown path: abuse reports, a staff tool to disable links (with audit logs), and fast cache invalidation.
• Unguessable codes (random, not sequential) so private links cannot be enumerated.
• Validate input: only http and https URLs, maximum length, no javascript: or data: schemes.
• Custom aliases: reserve system paths (/v1, /admin, /login), filter offensive or impersonating aliases.
• Authorisation: only owners can delete links or see their stats (object-level checks, article 28).
• Interstitial warning pages for links flagged as suspicious, rather than redirecting blindly.

"We're building a public URL shortener with about 100 million new links a month and a hundred times as many redirects, roughly 12,000 redirects a second at peak, with a p99 target of 50 ms and very high availability for redirects.

Links live in a table keyed by a 7-character random base-62 code; the primary key guarantees uniqueness, and the rare collision is retried. Custom aliases share the table. Creation takes an idempotency key, checks the URL against abuse lists, inserts, and writes the new link through to Redis so it works immediately.

Redirects go through stateless app servers that check a tiny in-process cache, then Redis, then read replicas, and return a 302, which we chose over 301 so we can count clicks and honour deletion and expiry. With a 95% hit rate the database sees about 600 reads a second. Clicks are published to a stream and aggregated asynchronously, so analytics can never slow or break redirects. Expiry is enforced on read and cleaned up by a background job.

The redirect path depends only on app servers, cache and replicas, so primary failover, analytics outages or creation problems don't stop redirects. If growth outpaces one primary, we shard by hash of the code. The biggest operational risk is abuse, handled by URL checks, rate limits and a takedown path."

Pastebin lets users store text (code snippets, logs, notes) and share it by a short link. It is the URL shortener plus a blob:

Changes to the design:

• Create: the client sends the text (for large pastes, uploads it directly to object storage with a presigned URL, as AWS documents for S3 (presigned URLs)); the API stores metadata (code, object_key, size, content_type, expires_at, visibility) in the database.
• Read: look up metadata (cached), then serve the content from object storage through a CDN, which absorbs the bandwidth. Public pastes can be cached at the edge with long TTLs because content is immutable (a paste is never edited; a new version is a new paste).
• Expiry: object-storage lifecycle rules delete expired content; the metadata row returns 410.
• Private pastes: short-lived signed URLs for content, plus authorisation on metadata.
• Burn after reading: a conditional update marks the paste consumed on first read (UPDATE ... WHERE consumed = false), and only the request that changed the row gets the content: a nice small exercise in concurrency (article 14).
• Abuse: pastes can contain leaked credentials or malicious content; add size limits, scanning, rate limits and a takedown process.

Writes: 500M / 2.6M s = about 190 per second; peak about 580 per secondReads: 20 x 500M = 10 billion per month = about 3,850 per second; peak about 11,500 per secondStorage: 500M x 12 months x 10 years = 60 billion links x 400 bytes = 24 TB (before replicas)

• Next: Design a Distributed Rate Limiter, the component that protects this shortener's creation endpoint.
• Then: Design a Distributed Cache, the component that makes its redirect path fast.