Defining Nonfunctional Requirements

Functional vs Nonfunctional requirements

• Functional requirements, what the app must do: the screens, the buttons, what each operation accomplishes.
• Nonfunctional requirements, the qualities the app must have: fast, reliable, secure, legally compliant, maintainable. Often unwritten because they seem obvious, but just as important. "An app that is unbearably slow or unreliable might as well not exist."

This chapter covers four nonfunctional requirements in depth (security is acknowledged but largely out of scope):

Fullscreen

---

Case study: Social Network Home Timelines

A deliberately simplified Twitter/X clone: users post messages and follow each other. Assumed numbers (used to reason about scale):

• 500 million posts/day = ~5,800 posts/sec on average, spiking to ~150,000/sec.
• Average user follows 200 people and has 200 followers, but the range is huge (celebrities like Barack Obama have 100M+ followers).

The data model

Three tables in a relational DB: users, posts, follows.

The core read operation is the home timeline, recent posts by the people you follow. Naively, a SQL query joins posts → follows → users, filters to the current user's followees, sorts by timestamp, and limits to the most recent ~1,000.

The problem: the naive read is far too expensive

• Posts should appear within ~5 seconds. The simplest way to keep timelines fresh is polling, the client re-runs the query every 5 seconds.
• With 10M users online, that is ~2 million timeline queries/sec.
• Each query merges recent posts from ~200 followed accounts → ~400 million lookups/sec on average, and far worse for users following tens of thousands of accounts.

The fix: materialize on write (fan-out)

Two improvements: (1) the server pushes new posts to online followers instead of clients polling, and (2) precompute each user's timeline so reads come from a cache.

Fullscreen

• Fan-out = the factor by which one request multiplies into downstream requests. Here, one post fans out to ~200 follower-timeline writes.
• This is materialization, and the timeline cache is a materialized view: precompute and store query results. It speeds up reads at the cost of more work on writes.
• Trade-off math: ~5,800 posts/sec × 200 followers ≈ just over 1 million timeline writes/sec, far cheaper than the 400 million read-time lookups/sec.
• Load spikes are absorbed by enqueuing timeline deliveries; reads stay fast because they come from the cache.

Two extreme cases

• Heavy followers: a user following huge numbers of high-volume accounts generates a high write rate to their timeline, but they won't read it all, so it's fine to drop some writes and show a sample.
• Celebrities: a post by someone with millions of followers means millions of timeline inserts, and dropping these is not OK. Common fix: store celebrity posts separately and merge them at read time with the materialized timeline. Handling celebrities still requires significant infrastructure.

the same data supports a read-heavy access pattern, and the right architecture pushes work from the hot read path onto the write path. This pull-vs-push tradeoff recurs throughout the book.

---

Describing Performance

Two main metric families:

• Response time, elapsed time from request to answer, as the client sees it (seconds/ms/µs).
• Throughput, requests per second, or data volume per second. For fixed hardware there is a maximum throughput.

In the case study: "posts/sec" and "timeline writes/sec" are throughput; "time to load the timeline" and "time until a post reaches followers" are response time.

Throughput and response time are linked (queueing)

As throughput rises toward the hardware's capacity, response time climbs sharply because of queueing: a new request waits while the CPU finishes earlier ones. Near the limit, queueing delays explode.

When an overloaded system won't recover (metastable failure): Past a tipping point a system can enter a vicious cycle and stay broken even after load drops, until it is reset.

Fullscreen

Defenses against this cycle:

• Exponential backoff (+ jitter), increase and randomize the wait between retries.
• Circuit breaker / token bucket, temporarily stop calling a service that's failing.
• Load shedding, server proactively rejects requests when near overload.
• Backpressure, server tells clients to slow down.
• Good queueing and load-balancing algorithm choices.

Which metric matters for what: Users care most about response time; throughput determines how much hardware you need (and therefore cost). A system is scalable if its max throughput can be significantly increased by adding resources.

Latency vs response time (precise vocabulary)

These terms are often blurred; the book is specific:

Fullscreen

• Response time, what the client sees; includes all delays anywhere in the system.
• Service time, duration the service is actively processing the request.
• Queueing delay, time spent waiting (e.g., for a CPU, or to send a response packet).
• Latency, catch-all for time a request is latent (not being actively processed). Network latency / delay is the travel time through the network.
• Head-of-line blocking, a few slow requests hold up the ones behind them; a server processes only a few things in parallel, so even fast requests get a slow overall response. Because of this, measure response time on the client side.

Response time varies request-to-request due to random factors: context switches, TCP retransmissions, garbage collection pauses, page faults, even mechanical vibration in the rack. Variation in delay is called jitter.

Average, median, and percentiles

Treat response time as a distribution, not a single number.

• Mean (average), useful for estimating throughput limits, but a poor "typical" measure (it hides how many users actually felt that delay).
• Median (p50), the halfway point; better for "how long does a typical user wait?"
• Tail latencies (p95, p99, p999), the slow end. Example: p95 = 1.5s means 5 in 100 requests take ≥ 1.5s.

Why tails matter (Amazon's reasoning): Amazon specifies internal targets at the 99.9th percentile, even though it affects only 1 in 1,000 requests, because the slowest requests often belong to the most valuable customers (they have the most data). Amazon found the 99.99th percentile too expensive to optimize for diminishing returns, since extreme percentiles are dominated by random events outside your control.

Latency impact stats cited online vary wildly and some are unreliable (e.g., fast pages correlating with lower conversion, because 404 pages load fastest). A cleaner Yahoo study (controlling for result quality) found 20–30% more clicks on fast searches when the gap was ≥1.25s.

Tail latency amplification

When one end-user request triggers multiple backend calls (even in parallel), the request is only as fast as the slowest call. The more backend calls, the higher the chance at least one is slow, so a small fraction of slow backend calls produces a larger fraction of slow end-user requests.

Fullscreen

SLOs, SLAs, and computing percentiles

• SLO (Service Level Objective), a target, e.g., median < 200ms, p99 < 1s, and ≥ 99.9% of valid requests non-error.
• SLA (Service Level Agreement), a contract specifying consequences if the SLO is missed (e.g., refunds).
• Computing percentiles efficiently: keep a rolling window; for scale use approximation libraries, HdrHistogram, t-digest, OpenHistogram, DDSketch.
• Critical gotcha: averaging percentiles is mathematically meaningless. To combine data across machines or time windows, add the histograms.

---

Reliability and Fault Tolerance

Reliability: continuing to work correctly even when things go wrong. "Working correctly" typically means: does what users expect, tolerates user mistakes/misuse, performs well enough under expected load, and prevents unauthorized access/abuse.

Fault vs Failure

Fullscreen

• Fault, one component stops working correctly (a disk dies, a machine crashes, a dependency goes down).
• Failure, the system as a whole stops providing the required service (it misses its SLO).
• Same event, different levels: a failed disk is a failure of the disk but only a fault to the larger system, which may tolerate it.

Fault tolerance

• A system is fault-tolerant if it keeps serving despite certain faults. A part that the system cannot tolerate losing is a single point of failure (SPOF).
• Tolerance is always bounded, e.g., survive up to 2 disk failures, or 1 of 3 nodes. You can't tolerate every fault (if Earth is swallowed by a black hole, you'd need web hosting in space).
• In the case study, if a fan-out machine crashes, a fault-tolerant design lets another machine take over without missing or duplicating posts, this is exactly-once semantics (Chapter 12).
• Fault injection / chaos engineering, deliberately trigger faults (e.g., randomly kill processes) to keep the fault-tolerance machinery exercised. Many critical bugs come from poor error handling, so continually testing it raises confidence.
• Prevention vs cure: we usually prefer tolerating faults, but for security prevention is better, a leaked secret can't be un-leaked. This book mostly deals with curable faults.

Hardware faults

Rare on a small system, but routine at scale:

Tolerating hardware faults with redundancy:

• Component redundancy: RAID disks, dual power supplies, hot-swappable CPUs, batteries + diesel generators.
• Redundancy works best when faults are independent, but real failures are often correlated (a whole rack or datacenter can go down).
• Cloud systems lean less on single-machine reliability and more on software-level fault tolerance across machines. Availability zones mark which resources are physically co-located (and thus likely to fail together).
• Surviving the loss of a whole machine/rack/zone also enables rolling upgrades, patch one node at a time with no downtime (Chapter 5).

Software faults

Unlike hardware faults, software faults are often highly correlated, every node runs the same code, so they share the same bug, causing many simultaneous failures. Examples:

• A bug that fails every node under specific conditions (the 2012 leap-second Linux/Java hang; a firmware bug that bricked certain SSDs after exactly 32,768 hours).
• A runaway process exhausting a shared resource (CPU, memory, disk, bandwidth, threads).
• A dependency that slows, hangs, or returns corrupted responses.
• Emergent behavior from interactions only seen in production.
• Cascading failures, one overloaded component drags down the next.

No silver bullet. Helps: questioning assumptions/interactions, thorough testing, process isolation, letting processes crash and restart, avoiding feedback loops like retry storms, and measuring/monitoring in production.

Humans and reliability

• Humans are creative and adaptive, but unpredictable. One study found operator configuration changes were the leading cause of outages; hardware faults caused only 10–25%.
• "Human error" is a symptom, not a cause. It points to a problem in the sociotechnical system. Blaming people is counterproductive.
• Technical safeguards: thorough testing (including property testing on random inputs), fast rollback, gradual rollouts, clear monitoring, observability, and interfaces that make the right thing easy and the wrong thing hard.
• Organizations under-invest because features win over resilience, so when a preventable mistake happens, the real issue is organizational priorities.
• Blameless postmortems: after an incident, share full details without fear of punishment, so the org learns. Be suspicious of simplistic conclusions ("Bob should've been careful" or "rewrite it in Haskell"); learn how the system really works from the people who run it.

How important is reliability, the Post Office Horizon scandal: From 1999–2019, hundreds of UK Post Office operators were wrongly convicted of theft/fraud because accounting software showed false shortfalls; many convictions were later overturned. English law assumed computers operate correctly unless proven otherwise. People were imprisoned, bankrupted, and some died by suicide. A stark reminder that unreliable software harms real people. Sometimes you may trade reliability for lower cost (e.g., an unproven prototype), but do it consciously, aware of the consequences.

---

Scalability

Scalability = a system's ability to cope with increased load. It is not a one-dimensional label, "X is scalable" is meaningless. Instead, ask:

• If the system grows in a particular way, what are our options for coping?
• How do we add resources to handle the extra load?
• Given current growth, when will we hit the limits of this architecture?

"You're not Google." For a young product with few users, the priority is simplicity and flexibility so you can adapt fast. Premature scalability work is wasted effort at best and a design straitjacket at worst. Worry about scale once you actually have load and know where the bottlenecks are.

Understanding load

You must quantify current load before reasoning about growth. Load is described by load parameters chosen for your app: requests/sec, GB of new data/day, checkouts/hour, peak simultaneous online users, read:write ratio, cache hit rate, followers per user, etc. Sometimes the average matters; sometimes a few extreme cases dominate.

Two ways to look at growth:

1. Increase load with fixed resources → how does performance degrade?
2. Increase load and keep performance fixed → how much must resources grow?

• Linear scalability, double the resources handle double the load at the same performance (good). Occasionally you do better (economies of scale); more often, cost grows faster than linearly.

Three architectures for adding hardware

Fullscreen

Shared-nothing has gained popularity: potential linear scaling, best price/performance hardware, easy to grow/shrink, fault tolerance across datacenters/regions. Some cloud-native DBs separate storage and compute, resembling shared-disk, but with a specialized storage API rather than a NAS/SAN abstraction, avoiding the old scalability problems.

Principles for scalability

• No "magic scaling sauce." Large-scale architecture is highly application-specific. A system for 100,000 requests/sec of 1 kB each looks nothing like one for 3 requests/min of 2 GB each, even at the same 100 MB/s throughput.
• Rethink each order of magnitude. An architecture for one load level usually won't survive 10×. Don't plan more than ~1 order of magnitude ahead.
• Break the system into independent components. The principle behind microservices, sharding, stream processing, and shared-nothing. The hard part is knowing where to draw the boundaries.
• Don't over-complicate. If a single-machine DB works, prefer it. Manual scaling can have fewer surprises than autoscaling when load is predictable. 5 services is simpler than 50. Good architectures mix approaches pragmatically.

---

Maintainability

Software doesn't wear out, but requirements evolve, platforms change, and bugs need fixing. Most of software's cost is ongoing maintenance, not initial development, fixing bugs, keeping it running, investigating failures, porting, adapting, repaying tech debt, adding features.

Maintenance is also a people problem: legacy systems may use outdated tech (mainframes, COBOL), and the institutional knowledge of why it was built that way is often lost. Every valuable system eventually becomes legacy, so design with maintenance in mind via three pillars:

Fullscreen

Operability

"Good operations can often work around bad software, but good software cannot run reliably with bad operations." Human processes matter as much as tools.

• Automation is essential at scale, but it's a double-edged sword: edge cases still need skilled humans, and an automated system gone wrong is harder to debug. More automation is not always better, find the sweet spot.
• Good systems support operability by: exposing key metrics + observability; avoiding dependence on individual machines; clear docs and a predictable operational model ("if I do X, Y happens"); good defaults with override ability; self-healing where appropriate plus manual control; predictable behavior with minimal surprises.

Simplicity

• Large systems drift into complexity, the "big ball of mud." Complexity slows everyone, raises cost, and breeds bugs through hidden assumptions and unexpected interactions.
• Simplicity is subjective (a simple interface over a complex implementation, vs. a simple implementation exposing detail, which is "simpler"?).
• Essential vs accidental complexity: essential is inherent to the problem; accidental comes only from poor tooling. Useful framing, but the line shifts as tooling evolves.
• Abstraction is the best tool for managing complexity, hide implementation behind a clean façade, reusable across applications, and quality improvements benefit everyone. Examples: high-level languages hide machine code; SQL hides on-disk structures, concurrency, and crash recovery. This book provides general-purpose abstractions (transactions, indexes, event logs); application-specific methods like design patterns and domain-driven design (DDD) sit on top of them.

Evolvability

• Requirements are in constant flux (new facts, new use cases, business shifts, new features, new platforms, regulation, growth).
• Agile gives organizational and technical tools (TDD, refactoring). The book calls agility at the data-system level evolvability.
• Evolvability is tied to simplicity and good abstractions: loosely coupled, simple systems are easier to change than tightly coupled, complex ones.
• A major obstacle to change is irreversibility. Migrating databases is far riskier if you can't switch back. Minimizing irreversibility improves flexibility, take irreversible actions very carefully.

---

Real-world examples and analogies

• Pull vs push timelines, restaurant à la carte vs buffet. Pull (à la carte) cooks your dish only when you order, expensive per request. Push (buffet) prepares everything in advance so serving is instant, but you do a lot of prep work, and a celebrity's "order" (millions of followers) would overwhelm the kitchen, so you handle those specially.
• Throughput vs response time, a highway. A near-empty highway lets you drive at full speed (low response time). As traffic approaches capacity, small additions cause huge slowdowns (queueing). Push past the limit and you get gridlock that persists even after cars leave, a metastable jam.
• Percentiles vs average, the commute. "Average commute is 30 minutes" hides that one day a month it's two hours. You plan your life around the bad days (p95/p99), not the average. SLAs are written in percentiles for the same reason.
• Tail latency amplification, the group photo. Everyone is ready except one person fumbling with their phone, the whole group waits for the slowest member. Add more people and the odds that someone is slow go up.
• Fault vs failure, a spare tire. A flat tire is a fault. With a spare (redundancy), you keep driving, tolerated. With no spare, the flat becomes a failure: the trip stops.
• Correlated software faults, a typo in a recipe. If every cook in the chain uses the same flawed recipe, they all ruin the dish at once. Hardware faults are like one oven breaking; software faults are the shared recipe.
• Blameless postmortems, the airline safety model. Aviation improved by investigating systems and procedures after incidents rather than punishing pilots, which is why people report problems honestly.
• Vertical vs horizontal scaling, one strong mover vs a moving crew. Vertical: hire one increasingly strong person (expensive, hits physical limits). Horizontal: hire more ordinary movers who coordinate (cheaper per unit, but now they must communicate).
• Accidental complexity, a tangled extension-cord pile. The task (power three lamps) is simple; the knot of cords is accidental complexity from bad tooling/organization, not the problem itself.
• Irreversibility, a one-way door. Some decisions are like a door that locks behind you; walk through them slowly. Reversible decisions are two-way doors you can make fast.

---

Cheat-sheet flashcards

Cover the answer and recall it.

• Functional vs nonfunctional requirement? → What the app does vs the qualities it must have.
• Four nonfunctional requirements in this chapter? → Performance, reliability, scalability, maintainability.
• Response time vs throughput? → Time per request (client-perceived) vs requests/data per second.
• What is fan-out? → The factor by which one request multiplies into downstream requests.
• What is a materialized view? → Precomputed, stored query results (speeds reads, costs more on writes).
• How is the celebrity fan-out problem solved? → Store celebrity posts separately, merge at read time.
• Why does response time spike near capacity? → Queueing, requests wait for a busy CPU.
• What is a metastable failure? → System stays overloaded even after load drops, until reset.
• Name two retry-storm defenses. → Exponential backoff (+jitter), circuit breaker (also load shedding, backpressure).
• Latency vs service time? → Latency = time not actively processed; service time = time actively processing.
• What is head-of-line blocking? → A few slow requests delay the ones queued behind them.
• p50 / p95 / p99 mean? → 50th/95th/99th percentile response times (median and tails).
• Why does Amazon target p99.9? → Slowest requests often belong to the most valuable (data-heavy) customers.
• Can you average percentiles? → No, it's meaningless; add the histograms instead.
• SLO vs SLA? → Objective (target) vs Agreement (contract with consequences).
• Fault vs failure? → A component stops working vs the whole system stops serving.
• What is a SPOF? → Single point of failure, a part the system can't tolerate losing.
• What is chaos engineering? → Deliberately injecting faults to test fault tolerance.
• Why are software faults worse than hardware faults? → They're highly correlated (same code, same bug everywhere).
• Leading cause of outages in one study? → Operator configuration changes.
• What is a blameless postmortem? → Learning from incidents without punishing individuals.
• Is "X is scalable" a meaningful statement? → No, scalability depends on how load grows.
• Load parameter? → A chosen metric describing load (req/s, read:write ratio, followers/user…).
• Linear scalability? → 2× resources handle 2× load at the same performance.
• Shared-memory / shared-disk / shared-nothing? → Vertical (one big machine) / shared disk array (NAS/SAN) / independent nodes (scale out).
• Three pillars of maintainability? → Operability, simplicity, evolvability.
• Big ball of mud? → A system mired in complexity.
• Essential vs accidental complexity? → Inherent to the problem vs caused by poor tooling.
• Best tool against complexity? → Abstraction.
• Why does irreversibility hurt evolvability? → You can't roll back, so changes are high-stakes.

---

Common interview questions

1. Design a Twitter-style home timeline. How do you make reads fast at scale? (Materialize on write / fan-out, cache timelines, handle celebrities by merging at read time, absorb spikes by enqueuing.)
2. What's the difference between response time and throughput, and how are they related? (Client-perceived time vs rate; near capacity, queueing makes response time spike.)
3. Why report percentiles instead of averages for latency? (Distribution, not a single number; tails affect real users; mean hides outliers.)
4. What is tail latency amplification and when does it bite? (Multiple backend calls per request; the slowest dominates; more calls → more slow requests.)
5. Explain SLO vs SLA. How would you define an SLO for a service? (Targets like p99 < 1s, ≥99.9% non-error; SLA adds consequences.)
6. What is a metastable failure, and how do you prevent retry storms? (Self-sustaining overload; backoff+jitter, circuit breakers, load shedding, backpressure.)
7. Differentiate fault and failure. How do you build fault tolerance? (Component vs whole-system; redundancy, removing SPOFs, software-level tolerance, chaos engineering.)
8. Why are software faults often more dangerous than hardware faults? (Correlation, same code, same bug across all nodes simultaneously.)
9. How should an organization handle operator mistakes that cause outages? (Treat human error as a system symptom; blameless postmortems; rollbacks, gradual rollouts, better interfaces.)
10. Compare vertical, shared-disk, and shared-nothing scaling. When would you choose each? (Scale up for simplicity/small scale; shared-nothing for linear scale and fault tolerance at cost of distributed complexity.)
11. Someone says "just make it scalable." What's wrong with that, and what would you ask? (Scalability isn't one-dimensional; ask about load parameters, growth shape, and where the bottleneck is.)
12. What does "maintainability" mean, and how do you design for it? (Operability, simplicity, evolvability; good abstractions, loose coupling, minimizing irreversibility.)

---

Key terms glossary

---