Skip to content

MigratableResolver

MigratableResolver is an abstraction of resolvers that allow Spark to migrate shuffle blocks.

Contract

getMigrationBlocks

getMigrationBlocks(
  shuffleBlockInfo: ShuffleBlockInfo): List[(BlockId, ManagedBuffer)]

Used when:

  • ShuffleMigrationRunnable is requested to run

getStoredShuffles

getStoredShuffles(): Seq[ShuffleBlockInfo]

Used when:

putShuffleBlockAsStream

putShuffleBlockAsStream(
  blockId: BlockId,
  serializerManager: SerializerManager): StreamCallbackWithID

Used when:

Implementations

Back to top