Learn how systems scale
A notebook for distributed systems. Each idea is taught with diagrams you step through, a small working copy in Go, and a quiz to help you remember it.
Three ways to use this site
Learn
One idea at a time. Read, step through the diagram, change the numbers, then test yourself.
6 topics in 10 tracks
Design
Real systems, designed from the requirements to the failure modes, in the order of an interview.
First case study is next
Build
Small working copies in Go, built one test at a time. See each change and its test output.
2 builds, all tests passing
The tracks
Ten tracks, from the numbers every engineer should know to security. Each finished topic is filled in. The roadmap shows the reading order.
- Start here
How to approach a system design problem, Numbers every engineer should know, Back-of-envelope estimation, Performance vs scalability, scaling up vs scaling out, Availability in numbers: nines, SLOs, error budgets
- Networking and communication
How a request travels, how services talk to each other, and how traffic is spread over servers.
- Storage and databases
Data models, how storage engines work inside, transactions, and the special stores for blobs, search and time series.
- Replication and partitioning
Consistent hashing
- Consistency and consensus
What "consistent" means, how machines agree on order and on a leader, and how transactions span services.
- Caching
Where to put a cache, how to keep it correct, and what happens when one key gets all the traffic.
- Messaging, streams and batch
Doing work later and elsewhere, delivery guarantees, logs, and processing data in large batches or as events arrive.
- Reliability and operations
Failing well, protecting a service from overload, finding failures, and recovering from disasters.
- Architecture patterns
How to divide a system into services, and the patterns that connect them.
- Security
Who the caller is, what they can do, and how services trust each other.