public abstract class UserDefinedType<UserType> extends DataType implements scala.Serializable
This interface allows a user to make their own classes more interoperable with SparkSQL;
e.g., by creating a UserDefinedType
for a class X, it becomes possible to create
a DataFrame
which has class X in the schema.
For SparkSQL to recognize UDTs, the UDT must be annotated with
SQLUserDefinedType
.
The conversion via serialize
occurs when instantiating a DataFrame
from another RDD.
The conversion via deserialize
occurs when reading from a DataFrame
.
Constructor and Description |
---|
UserDefinedType() |
Modifier and Type | Method and Description |
---|---|
int |
defaultSize()
The default size of a value of the UserDefinedType is 4096 bytes.
|
abstract UserType |
deserialize(java.lang.Object datum)
Convert a SQL datum to the user type
|
java.lang.String |
pyUDT()
Paired Python UDT class, if exists.
|
abstract java.lang.Object |
serialize(java.lang.Object obj)
Convert the user type to a SQL datum
|
java.lang.String |
serializedPyClass()
Serialized Python UDT class, if exists.
|
abstract DataType |
sqlType()
Underlying storage type for this UDT
|
abstract java.lang.Class<UserType> |
userClass()
Class object for the UserType
|
buildFormattedString, fromCaseClassString, fromJson, json, prettyJson, simpleString, typeName
public abstract DataType sqlType()
public java.lang.String pyUDT()
public java.lang.String serializedPyClass()
public abstract java.lang.Object serialize(java.lang.Object obj)
TODO: Can we make this take obj: UserType? The issue is in CatalystTypeConverters.convertToCatalyst, where we need to convert Any to UserType.
obj
- (undocumented)public abstract UserType deserialize(java.lang.Object datum)
public abstract java.lang.Class<UserType> userClass()
public int defaultSize()
defaultSize
in class DataType