Lens — Why
Every system you'll ever debug follows the same rules
Tools change. Frameworks get replaced. But the forces underneath — how computation distributes, how time misleads, how state complicates, how failures hide — those don't change. Each concept here applies regardless of which tool you're using.
Execution at Scale
Why parallel systems behave unpredictably. Why shuffles dominate cost. Why more nodes often make things worse.
Scaling Limits
Why designs that work today collapse at 10x. Where coordination becomes the bottleneck.
State & Memory
Why remembering things makes systems fragile. Why exactly-once is harder than it sounds.
Data Shape & Layout
Why physical layout matters more than logical schema. Why file format and partitioning decide what's fast and what's not.
Data Correctness
Why pipelines succeed while producing wrong data. Why logic bugs outlive outages.
Change & Evolution
Why modifying pipelines is more dangerous than building them. Why backfills and migrations cause outsized failures.
Distributed System Guarantees
Why no system optimizes everything at once. Where engineering judgment lives — between competing constraints.
System Failures
Why distributed systems fail partially and silently. Why retries amplify damage. Why detection is harder than prevention.
Time & Ordering
Why clocks disagree. Why late data breaks everything. Why real-time is a spectrum, not a feature.