DAGSchedulerEventProcessLoop¶

DAGSchedulerEventProcessLoop is an event processing daemon thread to handle DAGSchedulerEvents (on a separate thread from the parent DAGScheduler's).

DAGSchedulerEventProcessLoop is registered under the name of dag-scheduler-event-loop.

DAGSchedulerEventProcessLoop uses java.util.concurrent.LinkedBlockingDeque blocking deque that can grow indefinitely.

Creating Instance¶

DAGSchedulerEventProcessLoop takes the following to be created:

DAGScheduler

DAGSchedulerEventProcessLoop is created when DAGScheduler is created.

Processing Event¶

DAGSchedulerEvent	Event Handler
AllJobsCancelled	doCancelAllJobs
BeginEvent	handleBeginEvent
CompletionEvent	handleTaskCompletion
ExecutorAdded	handleExecutorAdded
ExecutorLost	handleExecutorLost
GettingResultEvent	handleGetTaskResult
JobCancelled	handleJobCancellation
JobGroupCancelled	handleJobGroupCancelled
JobSubmitted	handleJobSubmitted
MapStageSubmitted	handleMapStageSubmitted
ResubmitFailedStages	resubmitFailedStages
SpeculativeTaskSubmitted	handleSpeculativeTaskSubmitted
StageCancelled	handleStageCancellation
TaskSetFailed	handleTaskSetFailed
WorkerRemoved	handleWorkerRemoved

messageProcessingTime Timer¶

DAGSchedulerEventProcessLoop uses messageProcessingTime timer to measure time of processing events.