ShuffleReader¶

ShuffleReader is an abstraction of shuffle block readers that can read combined key-value records for a reduce task.

Contract¶

Reading Combined Records (for Reduce Task)¶

read(): Iterator[Product2[K, C]]

Used when:

• CoGroupedRDD, ShuffledRDD, and SubtractedRDD are requested to compute a partition (for a ShuffleDependency dependency)
• ShuffledRowRDD (Spark SQL) is requested to compute a partition

Implementations¶

• BlockStoreShuffleReader