SpillListener¶

SpillListener is a SparkListener that intercepts (listens to) the following events for detecting spills in jobs:

• onTaskEnd
• onStageCompleted

SpillListener is used for testing only.

Creating Instance¶

SpillListener takes no input arguments to be created.

SpillListener is created when TestUtils is requested to assertSpilled and assertNotSpilled.

onTaskEnd Callback¶

onTaskEnd(
  taskEnd: SparkListenerTaskEnd): Unit

onTaskEnd...FIXME

onTaskEnd is part of the SparkListener abstraction.

onStageCompleted Callback¶

onStageCompleted(
  stageComplete: SparkListenerStageCompleted): Unit

onStageCompleted...FIXME

onStageCompleted is part of the SparkListener abstraction.