Broadcast¶

Broadcast[T] is an abstraction of broadcast variables (with the value of type T).

Contract¶

Destroying Variable¶

doDestroy(
  blocking: Boolean): Unit

Destroys all the data and metadata related to this broadcast variable

Used when:

• Broadcast is requested to destroy

Unpersisting Variable¶

doUnpersist(
  blocking: Boolean): Unit

Deletes the cached copies of this broadcast value on executors

Used when:

• Broadcast is requested to unpersist

Broadcast Value¶

getValue(): T

Gets the broadcast value

Used when:

• Broadcast is requested for the value

Implementations¶

• TorrentBroadcast

Creating Instance¶

Broadcast takes the following to be created:

• Unique Identifier

Serializable¶

Broadcast is a Serializable (Java) so it can be serialized (converted to bytes) and send over the wire from the driver to executors.

Destroying¶

destroy(): Unit // (1)
destroy(
  blocking: Boolean): Unit

1. Non-blocking destroy (blocking is false)

destroy removes persisted data and metadata associated with this broadcast variable.

Unpersisting¶

unpersist(): Unit // (1)
unpersist(
  blocking: Boolean): Unit

1. Non-blocking unpersist (blocking is false)

unpersist...FIXME

Brodcast Value¶

value: T

value makes sure that it was not destroyed and gets the value.

Text Representation¶

toString: String

toString uses the id as follows:

Broadcast([id])

Validation¶

Broadcast is considered valid until destroyed.

Broadcast throws a SparkException (with the text representation) when destroyed but requested for the value, to unpersist or destroy:

Attempted to use [toString] after it was destroyed ([destroySite])