JobWaiter¶

JobWaiter is a JobListener to listen to task events and to know when all have finished successfully or not.

Creating Instance¶

JobWaiter takes the following to be created:

• DAGScheduler
• Job ID
• Total number of tasks
• Result Handler Function ((Int, T) => Unit)

JobWaiter is created when DAGScheduler is requested to submit a job or a map stage.

Scala Promise¶

jobPromise: Promise[Unit]

jobPromise is a Scala Promise that is completed when all tasks have finished successfully or failed with an exception.

taskSucceeded¶

taskSucceeded(
  index: Int,
  result: Any): Unit

taskSucceeded executes the Result Handler Function with the given index and result.

taskSucceeded marks the waiter finished successfully when all tasks have finished.

taskSucceeded is part of the JobListener abstraction.

jobFailed¶

jobFailed(
  exception: Exception): Unit

jobFailed marks the waiter failed.

jobFailed is part of the JobListener abstraction.