public class DelimitedInputWriter extends AbstractRecordWriter
bucketIds, totalBuckets
Modifier and Type | Method and Description |
---|---|
protected static LazySimpleSerDe |
createSerde(Table tbl,
HiveConf conf,
char serdeSeparator)
Creates LazySimpleSerde
|
Object |
encode(byte[] record)
Encode a record as an Object that Hive can read with the ObjectInspector associated with the
serde returned by
AbstractRecordWriter.getSerde() . |
protected ObjectInspector[] |
getBucketObjectInspectors() |
protected StructField[] |
getBucketStructFields() |
protected LazySimpleStructObjectInspector |
getRecordObjectInspector() |
AbstractSerDe |
getSerde()
Get the SerDe for the Objects created by
AbstractRecordWriter.encode(byte[]) . |
char |
getSerdeSeparator() |
protected ArrayList<String> |
getTableColumns() |
protected byte[] |
reorderFields(byte[] record) |
void |
write(long transactionId,
byte[] record)
Writes using a hive RecordUpdater
|
clear, closeBatch, flush, getBucket, getObjectInspectorsForBucketedCols, newBatch
public DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, StreamingConnection conn) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fields. nulls or empty
strings in the array indicates the fields to be skippeddelimiter
- input field delimiterendPoint
- Hive endpointConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnpublic DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, StreamingConnection conn) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fields. nulls or empty
strings in the array indicates the fields to be skippeddelimiter
- input field delimiterendPoint
- Hive endpointconf
- a Hive conf object. Can be null if not using advanced hive settings.ConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnpublic DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, char serdeSeparator, StreamingConnection conn) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fieldsdelimiter
- input field delimiterendPoint
- Hive endpointconf
- a Hive conf object. Set to null if not using advanced hive settings.serdeSeparator
- separator used when encoding data that is fed into the
LazySimpleSerde. Ensure this separator does not occur
in the field dataconn
- connection this Writer is to be used withConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnpublic DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
DelimitedInputWriter(String[], String, HiveEndPoint, StreamingConnection)
public DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, StreamingConnection)
public DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, char serdeSeparator) throws ClassNotFoundException, StreamingException
DelimitedInputWriter(String[], String, HiveEndPoint, HiveConf, char, StreamingConnection)
protected byte[] reorderFields(byte[] record) throws UnsupportedEncodingException
UnsupportedEncodingException
public void write(long transactionId, byte[] record) throws SerializationError, StreamingIOFailure
RecordWriter
transactionId
- the ID of the Txn in which the write occursrecord
- the record to be writtenSerializationError
StreamingIOFailure
public AbstractSerDe getSerde()
AbstractRecordWriter
AbstractRecordWriter.encode(byte[])
. This is public so that test
frameworks can use it.getSerde
in class AbstractRecordWriter
protected LazySimpleStructObjectInspector getRecordObjectInspector()
getRecordObjectInspector
in class AbstractRecordWriter
protected StructField[] getBucketStructFields()
getBucketStructFields
in class AbstractRecordWriter
protected ObjectInspector[] getBucketObjectInspectors()
getBucketObjectInspectors
in class AbstractRecordWriter
public Object encode(byte[] record) throws SerializationError
AbstractRecordWriter
AbstractRecordWriter.getSerde()
. This is public so that test frameworks can use it.encode
in class AbstractRecordWriter
record
- record to be deserializedSerializationError
protected static LazySimpleSerDe createSerde(Table tbl, HiveConf conf, char serdeSeparator) throws SerializationError
tbl
- SerializationError
- if serde could not be initializedpublic char getSerdeSeparator()
Copyright © 2021 The Apache Software Foundation. All rights reserved.