org.apache.hadoop.hive.ql.io.orc

Interface Writer

All Known Implementing Classes:

WriterImpl

public interface Writer

The interface for writing ORC files.

Method Summary

Methods

Modifier and Type	Method and Description
void	addRow(Object row) Add a row to the ORC file.
void	addUserMetadata(String key, ByteBuffer value) Add arbitrary meta-data to the ORC file.
void	appendStripe(byte[] stripe, int offset, int length, StripeInformation stripeInfo, OrcProto.StripeStatistics stripeStatistics) Fast stripe append to ORC file.
void	appendUserMetadata(List<OrcProto.UserMetadataItem> userMetadata) When fast stripe append is used for merging ORC stripes, after appending the last stripe from a file, this interface must be used to merge any user metadata.
void	close() Flush all of the buffers and close the file.
long	getNumberOfRows() Return the number of rows in file.
long	getRawDataSize() Return the deserialized data size.
long	writeIntermediateFooter() Write an intermediate footer on the file such that if the file is truncated to the returned offset, it would be a valid ORC file.

Method Detail

addUserMetadata

void addUserMetadata(String key,
                   ByteBuffer value)

Add arbitrary meta-data to the ORC file. This may be called at any point until the Writer is closed. If the same key is passed a second time, the second value will replace the first.

Parameters:

key - a key to label the data with.

value - the contents of the metadata.

addRow

void addRow(Object row)
            throws IOException

Add a row to the ORC file.

Parameters:

row - the row to add

Throws:

IOException

close

void close()
           throws IOException

Flush all of the buffers and close the file. No methods on this writer should be called afterwards.

Throws:

IOException

getRawDataSize

long getRawDataSize()

Return the deserialized data size. Raw data size will be compute when writing the file footer. Hence raw data size value will be available only after closing the writer.

Returns:

raw data size

getNumberOfRows

long getNumberOfRows()

Return the number of rows in file. Row count gets updated when flushing the stripes. To get accurate row count this method should be called after closing the writer.

Returns:

row count

writeIntermediateFooter

long writeIntermediateFooter()
                             throws IOException

Write an intermediate footer on the file such that if the file is truncated to the returned offset, it would be a valid ORC file.

Returns:

the offset that would be a valid end location for an ORC file

Throws:

IOException

appendStripe

void appendStripe(byte[] stripe,
                int offset,
                int length,
                StripeInformation stripeInfo,
                OrcProto.StripeStatistics stripeStatistics)
                  throws IOException

Fast stripe append to ORC file. This interface is used for fast ORC file merge with other ORC files. When merging, the file to be merged should pass stripe in binary form along with stripe information and stripe statistics. After appending last stripe of a file, use appendUserMetadata() to append any user metadata.

Parameters:

stripe - - stripe as byte array

offset - - offset within byte array

length - - length of stripe within byte array

stripeInfo - - stripe information

stripeStatistics - - stripe statistics (Protobuf objects can be merged directly)

Throws:

IOException

appendUserMetadata

void appendUserMetadata(List<OrcProto.UserMetadataItem> userMetadata)

When fast stripe append is used for merging ORC stripes, after appending the last stripe from a file, this interface must be used to merge any user metadata.

Parameters:

userMetadata - - user metadata