Distributed systems: DDIA, 6.824, and Jepsen

Distributed systems is where intuition goes to die: the network partitions, the clock lies, and the “impossible” interleaving ships to production. The materials below are the ones that build correct instincts.

The map

• Designing Data-Intensive Applications — Martin Kleppmann. The best systems book of the last decade: replication, partitioning, transactions, consistency models, and stream processing, each with its failure modes attached. Chapters 5–9 are the core; read them twice. His distributed systems lecture series (Cambridge) is the free video companion.

The practice

• MIT 6.824 / 6.5840 — all lectures and labs public. Implementing Raft (lab 2) is the rite of passage: nothing else teaches what “the leader might not know it’s been deposed” feels like. Budget real weeks; the labs’ test suites are adversarial in exactly the way networks are.
• Raft — read the paper alongside the lab, and use the visualization when the paper’s figure 2 stops making sense at 1am.

The reality check

• Jepsen — Kyle Kingsbury’s distributed-systems testing analyses: real databases, real partitions, real violated guarantees, written up with unusual rigor and wit. Reading five Jepsen reports recalibrates how much you trust vendor consistency claims — permanently. The companion consistency models map is the clearest reference on what linearizable/sequential/snapshot actually promise.
• Aphyr’s “Strong consistency models” — the essay version, worth reading before the map.

Five papers worth reading in full

• Time, Clocks, and the Ordering of Events — Lamport, 1978. Still the foundation.
• The Google File System and MapReduce — how the modern data stack started.
• Dynamo — the availability-first design whose ideas (consistent hashing, hinted handoff, vector clocks) are everywhere.
• Spanner — the opposite bet: global strong consistency via TrueTime.

Why it’s on this list

The durable-agents project on this site is a distributed-systems problem wearing an AI costume: checkpointing, exactly-once side effects under kill -9, idempotency keys. The concepts transferred one-for-one from this reading — which is the argument for learning the foundations even if you never build a database.