Skip to content

Configuration Properties of Spark Standalone

spark.deploy.spreadOut

Controls whether standalone Master should perform round-robin scheduling across worker nodes (spreading out each app among all the nodes) instead of trying to consolidate each app onto a small number of nodes

Default: true

spark.master.rest.enabled

Default: false

Used when:

  • Master standalone application is requested to onStart

spark.master.rest.port

Default: 6066

Used when:

  • Master standalone application is requested to onStart
  • StandaloneSubmitRequestServlet is requested to buildDriverDescription

spark.worker.resourcesFile

(internal) Path to a file containing the resources allocated to the worker. The file should be formatted as a JSON array of ResourceAllocation objects. Only used internally in standalone mode.

Default: (undefined)

Used when:

  • LocalSparkCluster is requested to start
  • Worker standalone application is launched
Back to top