Skip to content

BlockDataManager

BlockDataManager is an abstraction of block data managers that manage storage for blocks of data (aka block storage management API).

BlockDataManager uses BlockId to uniquely identify blocks of data and ManagedBuffer to represent them.

BlockDataManager is used to initialize a BlockTransferService.

BlockDataManager is used to create a NettyBlockRpcServer.

Contract

getHostLocalShuffleData

getHostLocalShuffleData(
  blockId: BlockId,
  dirs: Array[String]): ManagedBuffer

Used when:

getLocalBlockData

getLocalBlockData(
  blockId: BlockId): ManagedBuffer

Used when:

  • NettyBlockRpcServer is requested to receive a request (OpenBlocks and FetchShuffleBlocks)

getLocalDiskDirs

getLocalDiskDirs: Array[String]

Used when:

putBlockData

putBlockData(
  blockId: BlockId,
  data: ManagedBuffer,
  level: StorageLevel,
  classTag: ClassTag[_]): Boolean

Stores (puts) a block data (as a ManagedBuffer) for the given BlockId. Returns true when completed successfully or false when failed.

Used when:

putBlockDataAsStream

putBlockDataAsStream(
  blockId: BlockId,
  level: StorageLevel,
  classTag: ClassTag[_]): StreamCallbackWithID

Used when:

releaseLock

releaseLock(
  blockId: BlockId,
  taskContext: Option[TaskContext]): Unit

Used when:

Implementations

Back to top