Measured · 13–14 Nov 2026

Recorded evidence

Real Spark executions recorded on one host. These tables are not a simulator. Use the controls to inspect saved repetitions, then reproduce the same code locally.

Performance probes: one warm-up per variant, then three measured repeats in alternating order. Recovery: three mechanism traces without warm-up. The AQE one-byte counterexample has one measured repeat. Timed variants include plan construction, execution and Python/JVM transfer; startup and source-cache materialization are excluded. Resource Spill is intentionally threshold-triggered.

Experiment Show warm-upDownload JSON

Cards show median and min–max wall time. Each row below is an observation, not a production recommendation.

Inspect a run

Run group

Final partitioning and SQL / stage metrics

Task attempts and per-task metrics

Saved lineage / physical plans

Conditions and mechanism observations

Evidence files and method

Task counters are extracted from Spark TaskEnd events. Failed attempts are retained. Shuffle read bytes include local and remote reads; all remote reads in this local baseline are zero. Shuffle write records can represent batched serialized objects, especially in PySpark. Peak execution memory is a maximum per task, not process RSS.

Raw environment events and runtime logs are excluded; reports contain only allowlisted metrics and path-cleaned plans. Wall times vary with JVM warm-up, OS scheduling and other host activity. No confidence interval or cross-machine generalization is claimed.

Reproduce these reports Read executable source