KryoSerializer¶

KryoSerializer is a Serializer that uses the Kryo serialization library.

Creating Instance¶

KryoSerializer takes the following to be created:

• SparkConf

KryoSerializer is created when:

• SerializerManager is created
• SparkConf is requested to registerKryoClasses
• SerializerSupport (Spark SQL) is requested for a SerializerInstance

useUnsafe Flag¶

KryoSerializer uses the spark.kryo.unsafe configuration property for useUnsafe flag (initialized when KryoSerializer is created).

useUnsafe is used when KryoSerializer is requested to create the following:

• KryoSerializerInstance
• KryoOutput

Creating New SerializerInstance¶

newInstance(): SerializerInstance

newInstance is part of the Serializer abstraction.

newInstance creates a KryoSerializerInstance with this KryoSerializer (and the useUnsafe and usePool flags).

newKryoOutput¶

newKryoOutput(): KryoOutput

newKryoOutput...FIXME

newKryoOutput is used when:

• KryoSerializerInstance is requested for the output

newKryo¶

newKryo(): Kryo

newKryo...FIXME

newKryo is used when:

• KryoSerializer is requested for a KryoFactory
• KryoSerializerInstance is requested to borrowKryo

KryoFactory¶

factory: KryoFactory

KryoSerializer creates a KryoFactory lazily (on demand and once only) for internalPool.

KryoPool¶

KryoSerializer creates a custom KryoPool lazily (on demand and once only).

KryoPool is used when:

• pool
• setDefaultClassLoader

supportsRelocationOfSerializedObjects¶

supportsRelocationOfSerializedObjects: Boolean

supportsRelocationOfSerializedObjects is part of the Serializer abstraction.

supportsRelocationOfSerializedObjects creates a new SerializerInstance (that is assumed to be a KryoSerializerInstance) and requests it to get the value of the autoReset field.