01 / Names, bytes & authority
Prerequisites —
Storage / Layout / Commit / Read
Where bytes live.
How changes commit.
A foundation-first course in distributed databases and storage. Inspect Parquet, stop a real DataNode, and observe a transaction conflict across real database nodes.
Two routes · distinct responsibilities
Analytical filesSQL engineIceberg (optional)ParquetFiles / HDFS / Objects
Transactional key rangesSQL gatewayTransactions / MVCCReplicated rangesLSM engine
Compatible roles, not one mandatory HDFS → Parquet → Iceberg → database pipeline.
06Bilingual chapters
04Real executable profiles
0File uploads in inspector
Coverage: chapters 02 / 03 / 05 / 06 have executed profiles. 01 / 04 remain reading and reasoning. Iceberg uses a local SQLite catalog, not a distributed/cloud catalog. All cluster nodes share one laptop.
Follow the concept dependencies
02 / Parquet physical layout
Prerequisites 01
03 / HDFS blocks & recovery
Prerequisites 01–02
04 / WAL, LSM, MVCC & indexes
Prerequisites 01–03
05 / Cross-range transactions
Prerequisites 04
06 / Snapshots & evolution
Prerequisites 02, 04–05
Four experiments, inspectable evidence
Equal rows. Different requests.
Five layouts, 45 measured queries. Separate footer candidates from real engine requests and timing.
Inspect layout evidence →Readable and fully replicated?
Four DataNodes, RF=3, thirteen blocks. Stop one node, verify reads, then prove repair on surviving targets.
Inspect recovery evidence →One rule. Two stale decisions.
Three actual nodes and distinct range leaseholders. Commit a transfer, then observe a real 40001 conflict.
Inspect transaction evidence →A file is not a committed table.
Read real 3/5/6-row snapshots, preserve field identity, mix partition specs and exclude a valid uncommitted file.
Inspect snapshot evidence →Scope boundary: Streaming Lab follows data changes; Distributed Compute Lab studies execution. This course focuses on storage layout, commit, placement and reading.