Skip to content

BlockStoreClient

BlockStoreClient is an abstraction of block clients that can fetch blocks from a remote node (an executor or an external service).

BlockStoreClient is a Java Closeable.

Note

BlockStoreClient was known previously as ShuffleClient (SPARK-28593).

Contract

Fetching Blocks

void fetchBlocks(
  String host,
  int port,
  String execId,
  String[] blockIds,
  BlockFetchingListener listener,
  DownloadFileManager downloadFileManager)

Fetches blocks from a remote node (using DownloadFileManager)

Used when:

Shuffle Metrics

MetricSet shuffleMetrics()

Shuffle MetricsSet

Default: (empty)

Used when:

Implementations

Back to top