public class DelimitedInputWriter extends Object
Constructor and Description |
---|
DelimitedInputWriter(String[] colNamesForFields,
String delimiter,
HiveEndPoint endPoint)
Constructor.
|
DelimitedInputWriter(String[] colNamesForFields,
String delimiter,
HiveEndPoint endPoint,
HiveConf conf)
Constructor.
|
DelimitedInputWriter(String[] colNamesForFields,
String delimiter,
HiveEndPoint endPoint,
HiveConf conf,
char serdeSeparator)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
clear()
Clear bufferred writes.
|
void |
closeBatch()
Close the RecordUpdater.
|
protected LazySimpleSerDe |
createSerde(Table tbl,
HiveConf conf)
Creates LazySimpleSerde
|
void |
flush()
Flush records from buffer.
|
char |
getSerdeSeparator() |
protected ArrayList<String> |
getTableColumns() |
void |
newBatch(Long minTxnId,
Long maxTxnID)
Creates a new record updater for the new batch
|
protected byte[] |
reorderFields(byte[] record) |
void |
write(long transactionId,
byte[] record)
Writes using a hive RecordUpdater
|
public DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fields. nulls or empty
strings in the array indicates the fields to be skippeddelimiter
- input field delimiterendPoint
- Hive endpointConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnpublic DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fields. nulls or empty
strings in the array indicates the fields to be skippeddelimiter
- input field delimiterendPoint
- Hive endpointconf
- a Hive conf object. Can be null if not using advanced hive settings.ConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnpublic DelimitedInputWriter(String[] colNamesForFields, String delimiter, HiveEndPoint endPoint, HiveConf conf, char serdeSeparator) throws ClassNotFoundException, ConnectionError, SerializationError, InvalidColumn, StreamingException
colNamesForFields
- Column name assignment for input fieldsdelimiter
- input field delimiterendPoint
- Hive endpointconf
- a Hive conf object. Set to null if not using advanced hive settings.serdeSeparator
- separator used when encoding data that is fed into the
LazySimpleSerde. Ensure this separator does not occur
in the field dataConnectionError
- Problem talking to HiveClassNotFoundException
- Serde class not foundSerializationError
- Serde initialization/interaction failedStreamingException
- Problem acquiring file system path for partitionInvalidColumn
- any element in colNamesForFields refers to a non existing columnprotected byte[] reorderFields(byte[] record) throws UnsupportedEncodingException
UnsupportedEncodingException
public void write(long transactionId, byte[] record) throws SerializationError, StreamingIOFailure
RecordWriter
transactionId
- the ID of the Txn in which the write occursrecord
- the record to be writtenSerializationError
StreamingIOFailure
protected LazySimpleSerDe createSerde(Table tbl, HiveConf conf) throws SerializationError
tbl
- SerializationError
- if serde could not be initializedpublic char getSerdeSeparator()
public void flush() throws StreamingIOFailure
RecordWriter
flush
in interface RecordWriter
StreamingIOFailure
public void clear() throws StreamingIOFailure
RecordWriter
clear
in interface RecordWriter
StreamingIOFailure
public void newBatch(Long minTxnId, Long maxTxnID) throws StreamingIOFailure, SerializationError
newBatch
in interface RecordWriter
minTxnId
- smallest Txnid in the batchmaxTxnID
- largest Txnid in the batchStreamingIOFailure
- if failed to create record updaterSerializationError
public void closeBatch() throws StreamingIOFailure
RecordWriter
closeBatch
in interface RecordWriter
StreamingIOFailure
Copyright © 2017 The Apache Software Foundation. All rights reserved.