System Design Fundamentals for Interviews: Scalability, Trade-offs, and the Framework Interviewers Actually Grade

API Design Essentials

The API is the contract between a client and a system, and the shape of that contract has real consequences for performance, evolvability, and correctness under concurrent or repeated requests.

REST vs. RPC vs. GraphQL, at a Glance

None of the three is a universal upgrade over the others — REST's simplicity and cacheability suit a public-facing CRUD API well, gRPC's compact binary format and strict schema suit high-throughput internal service calls, and GraphQL's client-specified shape suits a mobile client stitching together data from several backend resources in a single round trip specifically because that client cares a lot about minimizing round trips.

Pagination: Offset vs. Cursor

• Offset-based pagination (?page=3&limit=20 or ?offset=40&limit=20) is simple to implement and lets a client jump to an arbitrary page directly. Its real weakness shows up when rows are being inserted or deleted while someone pages through results: an item deleted from an earlier page shifts every subsequent row's offset back by one, which can cause the next page fetched to skip a row or repeat one the client already saw.
• Cursor-based pagination returns an opaque pointer (a cursor — often an encoded value derived from the last row's sort key) alongside each page, and the client passes that cursor back to fetch the next page relative to it, rather than by numeric position. This stays correct under concurrent inserts and deletes, because "give me everything after this specific row" doesn't shift the way a numeric offset does — the trade is that a client can no longer jump straight to page 7 without walking through the cursors before it.

High-traffic, frequently-changing feeds (a social feed, a live activity log) consistently favor cursor-based pagination for exactly this reason — offset pagination's failure mode gets more likely to actually surface the more concurrent writes are happening, which is precisely when the feed is busiest.

Idempotency Keys

An HTTP POST isn't naturally idempotent — sending the same "charge this customer $50" request twice is supposed to charge them twice, that's the whole point of a create operation. That becomes a real problem the moment a network hiccups: a client sends a payment request, the request succeeds server-side, but the response gets lost in transit — from the client's point of view, this looks identical to the request never having arrived, and the natural instinct is to retry it.

An idempotency key solves this: the client generates a unique key (a UUID, typically) for a given logical operation and sends it along with the request. The server records which keys it has already processed and, if the same key shows up again, returns the original response instead of performing the operation a second time:

POST /charge
Idempotency-Key: 8f14e45f-ceea-467e-bd3a-...
{ "amount": 50, "customer": "cust_123" }

Server logic:
  if idempotency_key already seen:
      return the stored response from the first time it was seen
  else:
      perform the charge
      store (idempotency_key -> response)
      return response

This is exactly the idempotency requirement from the message-queue chapter, applied to synchronous APIs instead of queue consumers — the same underlying problem (a duplicate delivery of the "same" logical operation) shows up in both, and the same fix (make repeats a no-op) solves both.

Rate-Limiting Algorithms

Rate limiting caps how many requests a client can make in a given window, protecting the system from abuse or accidental overload. The specific algorithm changes how strictly — and how smoothly — that cap is enforced.

Token bucket — a bucket holds up to capacity tokens, refilling at a fixed rate; each request consumes one token, and a request with no tokens available is rejected (or delayed). This naturally allows short bursts up to the bucket's capacity while still enforcing a steady average rate over time:

function allow_request(bucket):
    now = current_time()
    elapsed = now - bucket.last_refill_time
    bucket.tokens = min(bucket.capacity, bucket.tokens + elapsed * bucket.refill_rate)
    bucket.last_refill_time = now
    if bucket.tokens >= 1:
        bucket.tokens -= 1
        return True   # allowed
    return False      # rejected

Leaky bucket — requests fill a fixed-size queue ("bucket") and are processed ("leak out") at a strictly constant rate, regardless of how bursty the incoming requests were. Unlike token bucket, it smooths bursts out into a steady stream rather than allowing them through immediately — the trade-off is added latency for requests waiting in the queue during a burst, in exchange for a perfectly even output rate.

Fixed window counter — count requests in a fixed time window (e.g., "the current minute") and reset the count to zero at each window boundary. Simple to implement, but has an edge-burst problem: a client can send its full limit right at the end of one window and its full limit again right at the start of the next, achieving nearly double the intended rate across that boundary.

Sliding window counter — instead of a hard reset at the window boundary, weight the previous window's count by how much of it still overlaps the current sliding window, giving a smoothed estimate that avoids the fixed-window's boundary-burst problem without the full cost of tracking every individual request timestamp:

function allow_request(current_count, previous_count, elapsed_into_current_window, window_size, limit):
    weight = (window_size - elapsed_into_current_window) / window_size
    estimated_count = previous_count * weight + current_count
    if estimated_count < limit:
        return True
    return False

Chapter 11's worked example picks this thread back up and asks the question this section deliberately doesn't answer: where does that counter actually live once there's more than one application server enforcing the same limit.