Package org.apache.hadoop.hive.ql.io.orc

The Optimized Row Columnar (ORC) File Format.

See: Description

Interface Summary

Interface	Description
BinaryColumnStatistics	Statistics for binary columns.
BooleanColumnStatistics	Statistics for boolean columns.
ColumnStatistics	Statistics that are available for all types of columns.
CompressionCodec
DateColumnStatistics	Statistics for DATE columns.
DecimalColumnStatistics	Statistics for decimal columns.
DirectDecompressionCodec
DoubleColumnStatistics	Statistics for float and double columns.
IntegerColumnStatistics	Statistics for all of the integer columns, such as byte, short, int, and long.
OrcFile.WriterCallback
OrcFile.WriterContext
OrcProto.BinaryStatisticsOrBuilder
OrcProto.BloomFilterIndexOrBuilder
OrcProto.BloomFilterOrBuilder
OrcProto.BucketStatisticsOrBuilder
OrcProto.ColumnEncodingOrBuilder
OrcProto.ColumnStatisticsOrBuilder
OrcProto.DateStatisticsOrBuilder
OrcProto.DecimalStatisticsOrBuilder
OrcProto.DoubleStatisticsOrBuilder
OrcProto.FooterOrBuilder
OrcProto.IntegerStatisticsOrBuilder
OrcProto.MetadataOrBuilder
OrcProto.PostScriptOrBuilder
OrcProto.RowIndexEntryOrBuilder
OrcProto.RowIndexOrBuilder
OrcProto.StreamOrBuilder
OrcProto.StringStatisticsOrBuilder
OrcProto.StripeFooterOrBuilder
OrcProto.StripeInformationOrBuilder
OrcProto.StripeStatisticsOrBuilder
OrcProto.TimestampStatisticsOrBuilder
OrcProto.TypeOrBuilder
OrcProto.UserMetadataItemOrBuilder
PositionProvider	An interface used for seeking to a row index.
Reader	The interface for reading ORC files.
RecordReader	A row-by-row iterator for ORC files.
StringColumnStatistics	Statistics for string columns.
StripeInformation	Information about the stripes in an ORC file that is provided by the Reader.
TimestampColumnStatistics	Statistics for Timestamp columns.
Writer	The interface for writing ORC files.

Class Summary

Class	Description
ConversionTreeReaderFactory	Factory for creating ORC tree readers.
FileDump	A tool for printing out the file structure of ORC files.
InStream
Metadata
MetadataReader
OrcFile	Contains factory methods to read or write ORC files.
OrcFile.ReaderOptions
OrcFile.WriterOptions	Options for creating ORC file writers.
OrcFileKeyWrapper	Key for OrcFileMergeMapper task.
OrcFileStripeMergeInputFormat
OrcFileStripeMergeRecordReader
OrcFileValueWrapper	Value for OrcFileMergeMapper.
OrcInputFormat	A MapReduce/Hive input format for ORC files.
OrcInputFormat.NullKeyRecordReader	Return a RecordReader that is compatible with the Hive 0.12 reader with NullWritable for the key instead of RecordIdentifier.
OrcNewInputFormat	An InputFormat for ORC files.
OrcNewOutputFormat	An OutputFormat that writes ORC files.
OrcNewSplit	OrcFileSplit.
OrcOutputFormat	A Hive OutputFormat for ORC files.
OrcProto
OrcProto.BinaryStatistics	Protobuf type orc.proto.BinaryStatistics
OrcProto.BinaryStatistics.Builder	Protobuf type orc.proto.BinaryStatistics
OrcProto.BloomFilter	Protobuf type orc.proto.BloomFilter
OrcProto.BloomFilter.Builder	Protobuf type orc.proto.BloomFilter
OrcProto.BloomFilterIndex	Protobuf type orc.proto.BloomFilterIndex
OrcProto.BloomFilterIndex.Builder	Protobuf type orc.proto.BloomFilterIndex
OrcProto.BucketStatistics	Protobuf type orc.proto.BucketStatistics
OrcProto.BucketStatistics.Builder	Protobuf type orc.proto.BucketStatistics
OrcProto.ColumnEncoding	Protobuf type orc.proto.ColumnEncoding
OrcProto.ColumnEncoding.Builder	Protobuf type orc.proto.ColumnEncoding
OrcProto.ColumnStatistics	Protobuf type orc.proto.ColumnStatistics
OrcProto.ColumnStatistics.Builder	Protobuf type orc.proto.ColumnStatistics
OrcProto.DateStatistics	Protobuf type orc.proto.DateStatistics
OrcProto.DateStatistics.Builder	Protobuf type orc.proto.DateStatistics
OrcProto.DecimalStatistics	Protobuf type orc.proto.DecimalStatistics
OrcProto.DecimalStatistics.Builder	Protobuf type orc.proto.DecimalStatistics
OrcProto.DoubleStatistics	Protobuf type orc.proto.DoubleStatistics
OrcProto.DoubleStatistics.Builder	Protobuf type orc.proto.DoubleStatistics
OrcProto.Footer	Protobuf type orc.proto.Footer
OrcProto.Footer.Builder	Protobuf type orc.proto.Footer
OrcProto.IntegerStatistics	Protobuf type orc.proto.IntegerStatistics
OrcProto.IntegerStatistics.Builder	Protobuf type orc.proto.IntegerStatistics
OrcProto.Metadata	Protobuf type orc.proto.Metadata
OrcProto.Metadata.Builder	Protobuf type orc.proto.Metadata
OrcProto.PostScript	Protobuf type orc.proto.PostScript
OrcProto.PostScript.Builder	Protobuf type orc.proto.PostScript
OrcProto.RowIndex	Protobuf type orc.proto.RowIndex
OrcProto.RowIndex.Builder	Protobuf type orc.proto.RowIndex
OrcProto.RowIndexEntry	Protobuf type orc.proto.RowIndexEntry
OrcProto.RowIndexEntry.Builder	Protobuf type orc.proto.RowIndexEntry
OrcProto.Stream	Protobuf type orc.proto.Stream
OrcProto.Stream.Builder	Protobuf type orc.proto.Stream
OrcProto.StringStatistics	Protobuf type orc.proto.StringStatistics
OrcProto.StringStatistics.Builder	Protobuf type orc.proto.StringStatistics
OrcProto.StripeFooter	Protobuf type orc.proto.StripeFooter
OrcProto.StripeFooter.Builder	Protobuf type orc.proto.StripeFooter
OrcProto.StripeInformation	Protobuf type orc.proto.StripeInformation
OrcProto.StripeInformation.Builder	Protobuf type orc.proto.StripeInformation
OrcProto.StripeStatistics	Protobuf type orc.proto.StripeStatistics
OrcProto.StripeStatistics.Builder	Protobuf type orc.proto.StripeStatistics
OrcProto.TimestampStatistics	Protobuf type orc.proto.TimestampStatistics
OrcProto.TimestampStatistics.Builder	Protobuf type orc.proto.TimestampStatistics
OrcProto.Type	Protobuf type orc.proto.Type
OrcProto.Type.Builder	Protobuf type orc.proto.Type
OrcProto.UserMetadataItem	Protobuf type orc.proto.UserMetadataItem
OrcProto.UserMetadataItem.Builder	Protobuf type orc.proto.UserMetadataItem
OrcRawRecordMerger	Merges a base and a list of delta files together into a single stream of events.
OrcRecordUpdater	A RecordUpdater where the files are stored as ORC.
OrcRecordUpdater.OrcOptions	An extension to AcidOutputFormat that allows users to add additional options.
OrcSerde	A serde class for ORC.
OrcSplit	OrcFileSplit.
OrcStruct
OrcUtils
Reader.Options	Options for creating a RecordReader.
ReaderImpl
RecordReaderFactory	Factory to create ORC tree readers.
RecordReaderUtils	Stateless methods shared between RecordReaderImpl and EncodedReaderImpl.
RecordReaderUtils.ByteBufferAllocatorPool
StripeStatistics
TreeReaderFactory	Factory for creating ORC tree readers.
TreeReaderFactory.BinaryTreeReader
TreeReaderFactory.BooleanTreeReader
TreeReaderFactory.BytesColumnVectorUtil
TreeReaderFactory.ByteTreeReader
TreeReaderFactory.CharTreeReader
TreeReaderFactory.DateTreeReader
TreeReaderFactory.DecimalTreeReader
TreeReaderFactory.DoubleTreeReader
TreeReaderFactory.FloatTreeReader
TreeReaderFactory.IntTreeReader
TreeReaderFactory.ListTreeReader
TreeReaderFactory.LongTreeReader
TreeReaderFactory.MapTreeReader
TreeReaderFactory.ShortTreeReader
TreeReaderFactory.StringDictionaryTreeReader	A reader for string columns that are dictionary encoded in the current stripe.
TreeReaderFactory.StringDirectTreeReader	A reader for string columns that are direct encoded in the current stripe.
TreeReaderFactory.StringTreeReader	A tree reader that will read string columns.
TreeReaderFactory.StructTreeReader
TreeReaderFactory.TimestampTreeReader
TreeReaderFactory.TreeReader
TreeReaderFactory.UnionTreeReader
TreeReaderFactory.VarcharTreeReader
VectorizedOrcInputFormat	A MapReduce/Hive input format for ORC files.
VectorizedOrcSerde	A serde class for ORC.
WriterImpl	An ORC file writer.

Enum Summary

Enum	Description
CompressionCodec.Modifier
CompressionKind	An enumeration that lists the generic compression algorithms that can be applied to ORC files.
OrcFile.CompressionStrategy
OrcFile.EncodingStrategy
OrcFile.OrcTableProperties	Enum container for all orc table properties.
OrcFile.Version	Create a version number for the ORC file format, so that we can add non-forward compatible changes in the future.
OrcFile.WriterVersion	Records the version of the writer in terms of which bugs have been fixed.
OrcProto.ColumnEncoding.Kind	Protobuf enum orc.proto.ColumnEncoding.Kind
OrcProto.CompressionKind	Protobuf enum orc.proto.CompressionKind
OrcProto.Stream.Kind	Protobuf enum orc.proto.Stream.Kind
OrcProto.Type.Kind	Protobuf enum orc.proto.Type.Kind

Package org.apache.hadoop.hive.ql.io.orc Description

The Optimized Row Columnar (ORC) File Format. This format:

• Decomposes complex column types into primitives
• Uses type-specific encoders for each column
  • Dictionary encodings for low cardinality columns
  • Run length encoding of data
  • variable length encoding of integers
• Divides file into large stripes
• Each stripe includes light-weight indexes that enable the reader to skip large sets of rows that don't satisfy the filter condition
• A file footer that contains meta-information about file
  • Precise byte range for each stripe
  • Type information for the file
  • Any user meta-information
• Seek to row number is implemented to support secondary indexes
• Support for additional generic compression: LZO, SNAPPY, ZLIB.

Format:

 HEADER (3 bytes) "ORC"
 STRIPE (0 or more stripes)
 FILE-FOOTER
 POST SCRIPT
 PS LENGTH (1 byte)
 
 

Stripe:

 INDEX-STREAM (0 or more)
 DATA-STREAM (0 or more)
 STRIPE-FOOTER