Skip to content

ShuffleExecutorComponents

ShuffleExecutorComponents is an abstraction of executor shuffle builders.

Contract

createMapOutputWriter

ShuffleMapOutputWriter createMapOutputWriter(
  int shuffleId,
  long mapTaskId,
  int numPartitions) throws IOException

Creates a ShuffleMapOutputWriter

Used when:

createSingleFileMapOutputWriter

Optional<SingleSpillShuffleMapOutputWriter> createSingleFileMapOutputWriter(
  int shuffleId,
  long mapId) throws IOException

Creates a SingleSpillShuffleMapOutputWriter

Default: empty

Used when:

initializeExecutor

void initializeExecutor(
  String appId,
  String execId,
  Map<String, String> extraConfigs);

Used when:

Implementations

Back to top