ShuffleWriteMetrics¶

ShuffleWriteMetrics is a ShuffleWriteMetricsReporter of metrics (accumulators) related to writing shuffle data (in shuffle map tasks):

• Shuffle Bytes Written
• Shuffle Write Time
• Shuffle Records Written

Creating Instance¶

ShuffleWriteMetrics takes no input arguments to be created.

ShuffleWriteMetrics is created when:

• TaskMetrics is created
• ShuffleExternalSorter is requested to writeSortedFile
• MapIterator (of BytesToBytesMap) is requested to spill
• ExternalAppendOnlyMap is created
• ExternalSorter is requested to spillMemoryIteratorToDisk
• UnsafeExternalSorter is requested to spill
• SpillableIterator (of UnsafeExternalSorter) is requested to spill

TaskMetrics¶

ShuffleWriteMetrics is available using TaskMetrics.shuffleWriteMetrics.

Serializable¶

ShuffleWriteMetrics is a Serializable (Java).