Skip to content

RangePartitioner

RangePartitioner is a Partitioner for bucketed partitioning.

RangePartitioner is used for sortByKey operator (among other uses).

Creating Instance

RangePartitioner takes the following to be created:

  • Number of Partitions
  • Key-Value RDD (RDD[_ <: Product2[K, V]])
  • ascending flag (default: true)
  • samplePointsPerPartitionHint (default: 20)

Number of Partitions

numPartitions: Int

numPartitions is the length of the rangeBounds array plus 1.

numPartitions is part of the Partitioner abstraction.

Partition for Key

getPartition(
  key: Any): Int

getPartition...FIXME

getPartition is part of the Partitioner abstraction.

Range Bounds

rangeBounds: Array[K]

rangeBounds is an Array[K]...FIXME

determineBounds Utility

determineBounds[K : Ordering : ClassTag](
  candidates: ArrayBuffer[(K, Float)],
  partitions: Int): Array[K]

determineBounds...FIXME