ShuffleWriteMetricsReporter¶

ShuffleWriteMetricsReporter is an abstraction of shuffle write metrics reporters.

Contract¶

decBytesWritten¶

decBytesWritten(
  v: Long): Unit

decRecordsWritten¶

decRecordsWritten(
  v: Long): Unit

incBytesWritten¶

incBytesWritten(
  v: Long): Unit

incRecordsWritten¶

incRecordsWritten(
  v: Long): Unit

incWriteTime¶

incWriteTime(
  v: Long): Unit

Used when:

• BypassMergeSortShuffleWriter is requested to write partition records and writePartitionedData
• UnsafeShuffleWriter is requested to mergeSpillsWithTransferTo
• DiskBlockObjectWriter is requested to commitAndGet
• TimeTrackingOutputStream is requested to write, flush, and close

Implementations¶

• ShuffleWriteMetrics
• SQLShuffleWriteMetricsReporter (Spark SQL)