public class ArrowColumnarBatchSerDe extends AbstractSerDe
ArrowWrapperWritable
, which doesn't support Writable.readFields(DataInput)
and Writable.write(DataOutput)
.
Followings are known issues of current implementation.
A list column cannot have a decimal column. UnionListWriter
doesn't have an
implementation for BaseWriter.ListWriter#decimal()
.
A union column can have only one of string, char, varchar fields at a same time. Apache Arrow
doesn't have string and char, so ArrowColumnarBatchSerDe
uses varchar to simulate
string and char. They will be considered as a same data type in
UnionVector
.
Timestamp with local timezone is not supported. VectorAssignRow
doesn't support it.Modifier and Type | Field and Description |
---|---|
static org.slf4j.Logger |
LOG |
configErrors
Constructor and Description |
---|
ArrowColumnarBatchSerDe() |
Modifier and Type | Method and Description |
---|---|
Object |
deserialize(org.apache.hadoop.io.Writable writable)
Deserialize an object out of a Writable blob.
|
ObjectInspector |
getObjectInspector()
Get the object inspector that can be used to navigate through the internal
structure of the Object returned from deserialize(...).
|
SerDeStats |
getSerDeStats()
Returns statistics collected when serializing
|
Class<? extends org.apache.hadoop.io.Writable> |
getSerializedClass()
Returns the Writable class that would be returned by the serialize method.
|
void |
initialize(org.apache.hadoop.conf.Configuration conf,
Properties tbl)
Initialize the HiveSerializer.
|
ArrowWrapperWritable |
serialize(Object obj,
ObjectInspector objInspector)
Serialize an object by navigating inside the Object with the
ObjectInspector.
|
getConfigurationErrors, initialize, shouldStoreFieldsInMetastore
public void initialize(org.apache.hadoop.conf.Configuration conf, Properties tbl) throws SerDeException
AbstractSerDe
initialize
in interface Deserializer
initialize
in interface Serializer
initialize
in class AbstractSerDe
conf
- System properties. Can be null in compile timetbl
- table propertiesSerDeException
public Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()
AbstractSerDe
getSerializedClass
in interface Serializer
getSerializedClass
in class AbstractSerDe
public ArrowWrapperWritable serialize(Object obj, ObjectInspector objInspector)
AbstractSerDe
serialize
in interface Serializer
serialize
in class AbstractSerDe
public SerDeStats getSerDeStats()
AbstractSerDe
getSerDeStats
in interface Deserializer
getSerDeStats
in interface Serializer
getSerDeStats
in class AbstractSerDe
public Object deserialize(org.apache.hadoop.io.Writable writable)
AbstractSerDe
deserialize
in interface Deserializer
deserialize
in class AbstractSerDe
writable
- The Writable object containing a serialized objectpublic ObjectInspector getObjectInspector()
AbstractSerDe
getObjectInspector
in interface Deserializer
getObjectInspector
in class AbstractSerDe
Copyright © 2022 The Apache Software Foundation. All rights reserved.