Skip to content

LocalCheckpointRDD

LocalCheckpointRDD[T] is a CheckpointRDD.

Creating Instance

LocalCheckpointRDD takes the following to be created:

LocalCheckpointRDD is created when:

Partitions

getPartitions: Array[Partition]

getPartitions is part of the RDD abstraction.

getPartitions creates a CheckpointRDDPartition for every input partition (index).

Computing Partition

compute(
  partition: Partition,
  context: TaskContext): Iterator[T]

compute is part of the RDD abstraction.

compute merely throws an SparkException (that explains the reason):

Checkpoint block [RDDBlockId] not found! Either the executor
that originally checkpointed this partition is no longer alive, or the original RDD is
unpersisted. If this problem persists, you may consider using `rdd.checkpoint()`
instead, which is slower than local checkpointing but more fault-tolerant."
Back to top