org.apache.hadoop.hive.ql.io.orc

Class OrcFile.WriterOptions

java.lang.Object

org.apache.hadoop.hive.ql.io.orc.OrcFile.WriterOptions

Enclosing class:

OrcFile

public static class OrcFile.WriterOptions
extends Object

Options for creating ORC file writers.

Method Summary

Methods

Modifier and Type	Method and Description
OrcFile.WriterOptions	blockPadding(boolean value) Sets whether the HDFS blocks are padded to prevent stripes from straddling blocks.
OrcFile.WriterOptions	blockSize(long value) Set the file system block size for the file.
OrcFile.WriterOptions	bloomFilterColumns(String columns) Comma separated values of column names for which bloom filter is to be created.
OrcFile.WriterOptions	bloomFilterFpp(double fpp) Specify the false positive probability for bloom filter.
OrcFile.WriterOptions	bufferSize(int value) The size of the memory buffers used for compressing and storing the stripe in memory.
OrcFile.WriterOptions	callback(OrcFile.WriterCallback callback) Add a listener for when the stripe and file are about to be closed.
OrcFile.WriterOptions	compress(CompressionKind value) Sets the generic compression that is used to compress the data.
OrcFile.WriterOptions	encodingStrategy(OrcFile.EncodingStrategy strategy) Sets the encoding strategy that is used to encode the data.
OrcFile.WriterOptions	fileSystem(org.apache.hadoop.fs.FileSystem value) Provide the filesystem for the path, if the client has it available.
OrcFile.WriterOptions	inspector(ObjectInspector value) A required option that sets the object inspector for the rows.
OrcFile.WriterOptions	paddingTolerance(float value) Sets the tolerance for block padding as a percentage of stripe size.
OrcFile.WriterOptions	rowIndexStride(int value) Set the distance between entries in the row index.
OrcFile.WriterOptions	stripeSize(long value) Set the stripe size for the file.
OrcFile.WriterOptions	version(OrcFile.Version value) Sets the version of the file that will be written.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

fileSystem

public OrcFile.WriterOptions fileSystem(org.apache.hadoop.fs.FileSystem value)

Provide the filesystem for the path, if the client has it available. If it is not provided, it will be found from the path.

stripeSize

public OrcFile.WriterOptions stripeSize(long value)

Set the stripe size for the file. The writer stores the contents of the stripe in memory until this memory limit is reached and the stripe is flushed to the HDFS file and the next stripe started.

blockSize

public OrcFile.WriterOptions blockSize(long value)

Set the file system block size for the file. For optimal performance, set the block size to be multiple factors of stripe size.

rowIndexStride

public OrcFile.WriterOptions rowIndexStride(int value)

Set the distance between entries in the row index. The minimum value is 1000 to prevent the index from overwhelming the data. If the stride is set to 0, no indexes will be included in the file.

bufferSize

public OrcFile.WriterOptions bufferSize(int value)

The size of the memory buffers used for compressing and storing the stripe in memory.

blockPadding

public OrcFile.WriterOptions blockPadding(boolean value)

Sets whether the HDFS blocks are padded to prevent stripes from straddling blocks. Padding improves locality and thus the speed of reading, but costs space.

encodingStrategy

public OrcFile.WriterOptions encodingStrategy(OrcFile.EncodingStrategy strategy)

Sets the encoding strategy that is used to encode the data.

paddingTolerance

public OrcFile.WriterOptions paddingTolerance(float value)

Sets the tolerance for block padding as a percentage of stripe size.

bloomFilterColumns

public OrcFile.WriterOptions bloomFilterColumns(String columns)

Comma separated values of column names for which bloom filter is to be created.

bloomFilterFpp

public OrcFile.WriterOptions bloomFilterFpp(double fpp)

Specify the false positive probability for bloom filter.

Parameters:

fpp - - false positive probability

Returns:

compress

public OrcFile.WriterOptions compress(CompressionKind value)

Sets the generic compression that is used to compress the data.

inspector

public OrcFile.WriterOptions inspector(ObjectInspector value)

A required option that sets the object inspector for the rows. Used to determine the schema for the file.

version

public OrcFile.WriterOptions version(OrcFile.Version value)

Sets the version of the file that will be written.

callback

public OrcFile.WriterOptions callback(OrcFile.WriterCallback callback)

Add a listener for when the stripe and file are about to be closed.

Parameters:

callback - the object to be called when the stripe is closed

Returns: