Day 1: CPU, RAM & the storage hierarchy
Why this is day one
Every phase in this course — containers, Kubernetes scheduling, database performance, even 'why is my API slow' — eventually traces back to these primitives. Skimming this phase is the single most common reason engineers plateau at mid-level: they can operate tools but can't explain *why* the tools behave the way they do.
The restaurant
CPU = chefs (do the actual work). RAM = the countertop (fast, but small — only what you're actively cooking with fits). Disk = the pantry (huge, but walking there takes time). Kernel = the kitchen manager, deciding which chef works on which order and when.
Kitchen manager (kernel) — decides who cooks when
Chefs
CPU cores
Countertop
RAM — fast, within reach
Pantry
Disk — slower, a walk away
The same analogy, laid out: chefs work fastest right at the countertop, and slow down every time they have to walk to the pantry.
The CPU
The CPU executes instructions — fetch, decode, execute, repeat, billions of times a second. A core is one independent execution unit; modern chips have multiple cores so multiple instruction streams run truly in parallel, not just interleaved. Each core has small, extremely fast cache memory (L1, L2, sometimes shared L3) that sits between it and RAM, holding data the CPU is likely to need again soon.
This matters later: Kubernetes CPU requests and limits (Phase 8) are ultimately the scheduler deciding how many core-seconds a container gets — the exact same resource this chapter describes.
RAM
RAM is volatile (it forgets everything on power loss) and byte-addressable — any location can be read or written directly, unlike disk which reads in blocks. It's dramatically faster than disk, but far smaller and far more expensive per gigabyte. This price/speed/size trade-off is why the 'storage hierarchy' exists at all: no single technology is fast, cheap, *and* large.
The storage hierarchy
Going from the CPU outward, each layer is roughly an order of magnitude bigger and slower than the last: CPU registers → L1/L2/L3 cache → RAM → SSD → spinning disk (HDD) → the network. Programs feel fast when the data they need is already close to the CPU, and feel slow the moment they have to reach further down this ladder.
L1 cache reference ~1 ns
L2 cache reference ~4 ns
Main memory (RAM) reference ~100 ns
SSD random read ~100 µs (~1,000x RAM)
HDD seek ~2-10 ms (~100,000x RAM)
Network round trip (same region) ~0.5 ms
Network round trip (cross-region) ~100-150 msThe same numbers, to scale (log axis) — each layer down the ladder is roughly an order of magnitude slower.
Why this ladder explains everything later
Redis is fast because it keeps everything in RAM instead of on disk. A database index is fast because it lets the query planner avoid reading the whole table off disk. A CDN is fast because it moves data physically closer, shortening the network hop. Same ladder, every time.
Key terms
- Core
- An independent execution unit within a CPU; multiple cores run instructions truly in parallel.
- Cache (L1/L2/L3)
- Small, very fast memory between the CPU and RAM that holds recently/likely-needed data.
- Volatile memory
- Memory that loses its contents when power is removed (RAM), as opposed to non-volatile storage (disk).
- Storage hierarchy
- The ladder of memory/storage types trading off speed, size, and cost: registers → cache → RAM → SSD → HDD → network.
Before moving on, you should be able to
Why does a database index make queries faster?
