Skip to content

JobWaiter

JobWaiter is a JobListener to listen to task events and to know when all have finished successfully or not.

Creating Instance

JobWaiter takes the following to be created:

  • DAGScheduler
  • Job ID
  • Total number of tasks
  • Result Handler Function ((Int, T) => Unit)

JobWaiter is created when DAGScheduler is requested to submit a job or a map stage.

Scala Promise

jobPromise: Promise[Unit]

jobPromise is a Scala Promise that is completed when all tasks have finished successfully or failed with an exception.

taskSucceeded

taskSucceeded(
  index: Int,
  result: Any): Unit

taskSucceeded executes the Result Handler Function with the given index and result.

taskSucceeded marks the waiter finished successfully when all tasks have finished.

taskSucceeded is part of the JobListener abstraction.

jobFailed

jobFailed(
  exception: Exception): Unit

jobFailed marks the waiter failed.

jobFailed is part of the JobListener abstraction.