Numbers every engineer should know

A design is a set of operations: read from memory, call a service, write to disk. Each one takes time. If you know about how long each takes, you can tell if a design is fast enough before you build it. Jeff Dean of Google calls this “an important skill: ability to estimate performance of a system design without actually having to build it”.

You do not need exact values. You need the order of magnitude of each operation, and the size of the jump between them. This lesson gives you both. It also measures the numbers on your own machine, because the famous table is from 2009.

The table

Jeff Dean showed this table in talks at Stanford (2007) and LADIS (2009). It is the table that most engineers know:

The values are from 2009. Some have changed a lot since then, and some have not changed at all. The next sections show which is which.

The jumps are what matter

The values in the table are hard to imagine. So make them human-sized: pretend that 1 ns is 1 second, and step through the ladder.

Remember three large jumps:

1. Cache to memory: about 100 times. A program that reads memory in a random order can be 100 times slower than one that reads it in order.
2. Memory to a network call: about 1,000 to 5,000 times. One extra call to another service costs more than a very large amount of computation.
3. Inside a datacenter to across an ocean: about 300 times. A request that crosses a continent pays for that distance on every round trip.

What changed since 2009

Hardware improved, but not every rung moved. Colin Scott keeps an interactive version of the table, with a source for each value. It also models how each value changes by year. Here is what changed and what did not:

So two old rules are no longer true. First, “a disk seek takes 10 ms” is true only for spinning disks. Second, the distance between memory and storage is now much smaller. But the distance between computers in different places is the same as in 2009, and it always will be.

Measure it yourself

A table from someone else’s machine is a starting point. The best numbers are the ones you measure. The latency-numbers build has small Go benchmarks for six rungs of the ladder. Each benchmark runs 7 times and keeps the median.

// Package latency measures a few rungs of the latency ladder on the machine that runs it.
// Each function returns the median of several runs, because one run is easy to disturb.
package latency

import (
	"bufio"
	"io"
	"math/rand/v2"
	"net"
	"os"
	"path/filepath"
	"slices"
	"sync"
	"time"
)

const runs = 7

func median(f func() time.Duration) time.Duration {
	ds := make([]time.Duration, runs)
	for i := range ds {
		ds[i] = f()
	}
	slices.Sort(ds)
	return ds[runs/2]
}

// chase builds a random cycle over n slots and follows it, so every load depends on the
// previous one. The CPU cannot prefetch or overlap the loads, so the time per step is the
// real latency of one memory access at that working-set size.
func chase(bytes, steps int) time.Duration {
	n := bytes / 8
	next := make([]int32, n)
	perm := rand.Perm(n)
	for i := range n {
		next[perm[i]] = int32(perm[(i+1)%n])
	}
	var p int32
	for range min(n, steps) { // warm up
		p = next[p]
	}
	start := time.Now()
	for range steps {
		p = next[p]
	}
	d := time.Since(start)
	sink = int(p)
	return d / time.Duration(steps)
}

var sink int

// L1Hit is one dependent load from a 16 KB working set, which fits in the L1 data cache.
func L1Hit() time.Duration {
	return median(func() time.Duration { return chase(16<<10, 20_000_000) })
}

// RandomMemoryRead is one dependent load from a 512 MB working set, which is far larger than
// every cache, so almost every load goes to main memory. The cycle is built once.
func RandomMemoryRead() time.Duration {
	memOnce.Do(func() { memResult = median(func() time.Duration { return chase(512<<20, 4_000_000) }) })
	return memResult
}

var (
	memOnce   sync.Once
	memResult time.Duration
)

// SequentialMemoryReadMB is the time to read 1 MB that is already in memory, in order.
func SequentialMemoryReadMB() time.Duration {
	buf := make([]byte, 64<<20)
	for i := range buf {
		buf[i] = byte(i)
	}
	return median(func() time.Duration {
		var sum uint64
		start := time.Now()
		for i := 0; i < len(buf); i += 8 {
			sum += uint64(buf[i])
		}
		d := time.Since(start)
		sink = int(sum)
		return d / 64
	})
}

// MutexLockUnlock is one Lock and Unlock of a sync.Mutex that no other goroutine holds.
func MutexLockUnlock() time.Duration {
	var mu sync.Mutex
	return median(func() time.Duration {
		const n = 10_000_000
		start := time.Now()
		for range n {
			mu.Lock()
			mu.Unlock()
		}
		return time.Since(start) / n
	})
}

// LoopbackRoundTrip sends a 64-byte message to a local TCP echo server and waits for the
// answer. It measures the operating system's network path with no physical network.
func LoopbackRoundTrip() (time.Duration, error) {
	ln, err := net.Listen("tcp", "127.0.0.1:0")
	if err != nil {
		return 0, err
	}
	defer ln.Close()
	go func() {
		c, err := ln.Accept()
		if err != nil {
			return
		}
		defer c.Close()
		_, _ = io.Copy(c, c)
	}()
	c, err := net.Dial("tcp", ln.Addr().String())
	if err != nil {
		return 0, err
	}
	defer c.Close()
	c.(*net.TCPConn).SetNoDelay(true)
	msg := make([]byte, 64)
	got := make([]byte, 64)
	r := bufio.NewReader(c)
	trip := func() error {
		if _, err := c.Write(msg); err != nil {
			return err
		}
		_, err := io.ReadFull(r, got)
		return err
	}
	for range 1000 { // warm up the connection
		if err := trip(); err != nil {
			return 0, err
		}
	}
	var failed error
	d := median(func() time.Duration {
		const n = 5000
		start := time.Now()
		for range n {
			if err := trip(); err != nil {
				failed = err
			}
		}
		return time.Since(start) / n
	})
	return d, failed
}

// SyncedWrite4KB writes 4 KB to a file and calls fsync, so the data must reach the storage
// device before the call returns. This is the cost that a database pays to make a commit durable.
func SyncedWrite4KB(dir string) (time.Duration, error) {
	f, err := os.Create(filepath.Join(dir, "sync.bin"))
	if err != nil {
		return 0, err
	}
	defer f.Close()
	block := make([]byte, 4096)
	var failed error
	d := median(func() time.Duration {
		const n = 50
		start := time.Now()
		for range n {
			if _, err := f.Write(block); err != nil {
				failed = err
			}
			if err := f.Sync(); err != nil {
				failed = err
			}
		}
		return time.Since(start) / n
	})
	return d, failed
}

// SequentialFileReadMB reads a 64 MB file in order and returns the time for each MB. The file
// was just written, so the operating system serves it from its page cache in memory.
func SequentialFileReadMB(dir string) (time.Duration, error) {
	path := filepath.Join(dir, "seq.bin")
	if err := os.WriteFile(path, make([]byte, 64<<20), 0o600); err != nil {
		return 0, err
	}
	buf := make([]byte, 1<<20)
	var failed error
	d := median(func() time.Duration {
		f, err := os.Open(path)
		if err != nil {
			failed = err
			return 0
		}
		defer f.Close()
		start := time.Now()
		for {
			_, err := io.ReadFull(f, buf)
			if err == io.EOF || err == io.ErrUnexpectedEOF {
				break
			}
			if err != nil {
				failed = err
				break
			}
		}
		return time.Since(start) / 64
	})
	return d, failed
}

=== RUN   TestCacheHitIsFasterThanRandomMemoryRead
    measure_test.go:15: cache hit (pointer chase, 16 KB): 1ns
    measure_test.go:16: random main-memory read (pointer chase, 512 MB): 137ns
--- PASS: TestCacheHitIsFasterThanRandomMemoryRead
=== RUN   TestSequentialMemoryReadPerMB
    measure_test.go:24: read 1 MB sequentially from memory: 35.995µs
--- PASS: TestSequentialMemoryReadPerMB
=== RUN   TestMutexLockUnlock
    measure_test.go:32: mutex lock + unlock (no contention): 10ns
--- PASS: TestMutexLockUnlock
=== RUN   TestLoopbackRoundTrip
    measure_test.go:43: TCP round trip on localhost (64-byte message): 15.247µs
--- PASS: TestLoopbackRoundTrip
=== RUN   TestDiskSyncedWrite
    measure_test.go:54: write 4 KB and fsync to SSD: 296.364µs
--- PASS: TestDiskSyncedWrite
=== RUN   TestSequentialDiskReadPerMB
    measure_test.go:65: read 1 MB sequentially from a file (OS page cache): 33.825µs
--- PASS: TestSequentialDiskReadPerMB
=== RUN   TestLadderOrder
    measure_test.go:81: ratio: memory read / cache hit = 137×
    measure_test.go:82: ratio: localhost round trip / memory read = 112×
--- PASS: TestLadderOrder
PASS
ok  	shardedmind/builds/latency-numbers/01-measure

The test output tab shows a run on the machine that built this site, so its numbers can differ. The numbers below were measured on an Apple M4 Pro laptop in October 2026:

cache hit (pointer chase, 16 KB): 1ns
random main-memory read (pointer chase, 512 MB): 118ns
mutex lock + unlock (no contention): 2ns
read 1 MB sequentially from memory: 32.6µs
TCP round trip on localhost (64-byte message): 17.714µs
write 4 KB and fsync to SSD: 3.981955ms
ratio: memory read / cache hit = 118×

Compare these results with the 2009 table:

• Memory is still about 100 ns. The random read took 118 ns, so the 2009 value is still correct. A cache hit took about 1 ns, so the jump is 118 times.
• Sequential memory reads are much faster. 1 MB took 32.6 µs, not 250 µs. Memory bandwidth increased about 8 times.
• A mutex is much cheaper when no one else holds it. It took 2 ns, not 100 ns. Go’s sync.Mutex does not call the operating system when the lock is free. The 2009 value is closer to the cost of a lock that other threads also want.
• A durable write is slow. A 4 KB write with fsync took about 4 ms. On macOS, Go’s fsync asks the drive to write its own cache to permanent storage. That is the true cost of a write that survives a power failure. A database pays this cost on each commit, unless it groups many commits into one write.

The localhost round trip (about 18 µs) does not use a real network. It shows only the cost of the operating system and the TCP code. A real round trip in a datacenter adds the time on the wire and in the network switches.

Use the numbers

Jeff Dean used the table to compare two designs for a page of 30 image thumbnails. In design 1, the server reads the images from disk one after the other. In design 2, it reads them all at the same time. Change the values and compare:

The arithmetic takes one minute and shows a 30 times difference. Now set the values for an SSD. The serial design becomes fast enough, and the best choice changes. The method stays the same, but the numbers change the answer.

When one slow server makes the request slow

Averages hide a problem. Many requests ask several servers at the same time and wait for all of them. Then the slowest server decides the time of the whole request.

Dean and Barroso give an example in The Tail at Scale. One server answers 99% of requests in 10 ms, and 1% in 1 second. A user request asks 100 such servers at the same time. Then 63% of user requests take more than 1 second.

The calculation is short. Each server is fast with a probability of 0.99. All 100 servers are fast with a probability of 0.99¹⁰⁰, which is about 0.37. So 63% of requests wait for at least one slow server.

This is why designs measure the 99th percentile (p99), not the average. The 99th percentile is the time that 99% of requests are faster than. The paper gives one fix: send a second copy of a slow request to another server, and use the first answer. The paper calls this a hedged request.

Recap

• Learn the order of magnitude of each operation, and the jumps between them.
• Cache to memory is about 100 times. Memory to a network call is about 1,000 to 5,000 times. A round trip across an ocean is about 150 ms, and it does not improve.
• Since 2009, storage became about 600 times faster (SSDs), and memory bandwidth increased. Memory latency and the speed of light did not change.
• A durable write (fsync) costs milliseconds. Databases group commits to share this cost.
• When a request waits for many servers, the slowest one decides its time. Measure the p99, not only the average.
• When you can, measure on your own hardware.