Diagnoses Spark data skew from the job's own event log - no cluster, no JVM - and tells key skew apart from a straggler by comparing task time against bytes read. Then measures the fixes against each other on real Spark 4: across 16 cells, salting won 0. Broadcast 6.5x, AQE 1.5x, salting 0.81x.
python benchmarking big-data apache-spark distributed-computing pyspark data-engineering performance-tuning profiling shuffle query-optimization observability salting spark4 broadcast-join adaptive-query-execution data-skew spark-event-log
-
Updated
Aug 19, 2026 - Python