How to approach a system design problem

A system design question is short. “Design a URL shortener.” “Design a chat system.” The answer is not short. Many people draw boxes at once and then get lost. This lesson gives you a method that works for any design question, at work or in an interview.

The method has one main idea. The requirements give the numbers, and the numbers decide the design. You do not choose a database because it is popular. You choose it because the numbers show what the data needs.

Why design needs a method

A design question has no single correct answer. It has many good answers, and each one fits a different set of requirements. So the first job is to find out which requirements apply. Without them, you cannot say if a design is good.

Google’s SRE teams call their method non-abstract large system design (NALSD). A design is non-abstract when it says how many machines it needs and what each one does. In the words of the SRE Workbook, a design must “turn a whiteboard design into concrete estimates of resources”. A design with no numbers often fails when it meets real hardware.

The five steps

Each step uses the result of the step before it. Step through them:

The five steps of a design, and what goes back

Step 1 of 7Expand

Requirements come first. Agree what the system must do, and how well. For example: 100 million users, a read takes under 100 ms, and a write is never lost.

Step 1 of 7BackNext step

1. Requirements

There are two kinds of requirement:

• Functional requirements say what the system does. “A user can upload a file.” “A user can share a link.”
• Non-functional requirements say how well it does it. “A page loads in under 200 ms for 99% of requests.” “The service is available 99.9% of the time.” “An upload is never lost.”

The non-functional requirements decide most of the design. Two systems with the same functions can need very different designs. A photo service for 1,000 users and one for 1 billion users do the same thing, but almost nothing else is the same.

Write each non-functional requirement as a number with a condition. The SRE book shows this form: “99% of Get RPC calls will complete in less than 100 ms.” A requirement like “fast” or “reliable” cannot be tested, so it cannot guide the design.

Also say what is out of scope. In an interview, this saves time. At work, it stops the project from growing without control.

2. Estimates

Turn the requirements into a small set of numbers:

Number	Why it matters
Requests each second, at the average and at the peak	How many servers you need
Reads for each write	Where a cache helps, and how to store the data
Bytes stored each day, and in total	How much storage, and when to split the data
Bytes moved each second	Network limits, and the cost of copies between regions
Latency target for each request	How many network calls and disk reads each request can make

Here is an example. A paste service gets 10 million new pastes a day. Change the numbers and see which resource becomes the problem:

From requirements to numbers, for a paste service

Users upload text and get a link. Other users read it with the link. There are 10 million new pastes a day. Is one database server enough for five years of data?

New pastes each dayMSize of one pasteKBReads for each writecountPeak traffic compared with the averagecountYears to keep the datacountCopies of the datacount

Assumptions

• Each paste has 10 KB of text and metadata, on average.
• Readers open each paste 10 times, so there are 10 reads for each write.
• Traffic at the busiest time of day is about 3 times the average.
• Each paste is kept for five years. The database stores 3 copies of the data.

Make your own guess first. Then show the steps one at a time.

Show the first stepShow all steps

This estimate shows the main purpose of the step. The traffic looked like the hard part, but the storage is the real problem. Without the numbers, you could spend all your time on the wrong part of the design. The next two lessons teach the numbers and the method in detail.

3. API and data

Write the main operations as an API: their inputs, their outputs and their errors. Keep it small. For the paste service, two operations are enough:

POST /pastes        body: text          → 201 { id }
GET  /pastes/{id}                       → 200 text | 404

Then write the data that the system stores and how it is read. These access patterns decide the storage. The paste service reads one item by its key, and never searches inside the text. So a key-value store is a good fit, and a full-text index is not necessary.

4. The design

Now draw the boxes: clients, load balancers, services, caches, databases, queues, object storage. For each box, say what it does and why it is there. Start with the simplest design that meets the numbers. Add a part only when a number or a requirement needs it.

Jeff Dean gives this rule: you must be able to “estimate performance of a system design without actually having to build it”. So for each box, know how many requests it gets and how long each one takes.

5. Deep dives

Choose the two or three hardest parts, and go into detail. Good choices are the parts with the highest load, the most data, or the most complex failure. For example: how do you make IDs that never repeat? What happens when the cache fails? How do you split the data over many nodes?

A deep dive often shows a problem in the design. Then you go back and change the design. This is the normal process. The SRE Workbook says that in practice, “we bounce around between the questions and phases”.

Four questions for every design

The NALSD method asks four questions of each version of a design. They are a good check at the end of any step:

Question	What it checks
Is it possible?	Can you build it with no “magic”, if resources were not a limit?
Can we do better?	Is it as simple as it can be? Is there a faster or smaller way?
Is it feasible?	Does it fit the real limits: money, hardware, time?
Is it resilient?	What happens when a machine, a disk or a datacenter fails?

The SRE Workbook works through a full example: a system that measures ad clicks for 500,000 search queries each second. It starts with one machine, finds that one machine needs 2,500 disks, and changes the design. Each version answers the four questions again. That example is a good model of this method.

Common mistakes

• Drawing before you ask. Without requirements, you cannot tell a good design from a bad one.
• No numbers. “We add a cache” is not a design. “The cache holds the 20% of pastes that get 80% of reads, which is 400 GB” is a design.
• Designing for the average. Traffic at the busiest hour is often 2 to 10 times the average. Design for the peak.
• Choosing technology first. Choose the data model and the access patterns first. Then choose a product that fits them.
• Ignoring failure. Every part fails at some time. Say what happens when it does.
• Not going back. If a deep dive shows a problem, change the design. Do not defend a design that the numbers do not support.

Recap

• The requirements give the numbers, and the numbers decide the design.
• Write functional requirements (what the system does) and non-functional requirements (how well), each as a number with a condition.
• Five steps: requirements, estimates, API and data, design, deep dives. Going back is normal.
• Access patterns (how the data is read and written) decide the storage, not popularity.
• NALSD (non-abstract large system design) asks four questions of each design: is it possible, can we do better, is it feasible, is it resilient?

Check what you learned

Question 1 of 8

An interviewer says: "Design a photo sharing service." What should you do first?

• ADraw the main boxes, then ask about the details
• BAsk how many users there are, how many uploads and reads there are, and how long photos are kept
• CChoose a database that is known to scale well
• DSay which cloud provider you would use

Check