org.apache.hadoop.hive.ql.exec.vector

Class VectorizedRowBatchCtx

java.lang.Object

org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatchCtx

public class VectorizedRowBatchCtx
extends Object

Context for Vectorized row batch. this calss does eager deserialization of row data using serde in the RecordReader layer. It has supports partitions in this layer so that the vectorized batch is populated correctly with the partition column.

Constructor Summary

Constructors

Constructor and Description
VectorizedRowBatchCtx() Constructor for VectorizedRowBatchCtx
VectorizedRowBatchCtx(StructObjectInspector rawRowOI, StructObjectInspector rowOI, Deserializer deserializer, Map<String,Object> partitionValues, Map<String,PrimitiveObjectInspector.PrimitiveCategory> partitionTypes) Constructor for VectorizedRowBatchCtx

Method Summary

Methods

Modifier and Type	Method and Description
void	addPartitionColsToBatch(VectorizedRowBatch batch) Add the partition values to the batch
void	addRowToBatch(int rowIndex, org.apache.hadoop.io.Writable rowBlob, VectorizedRowBatch batch, org.apache.hadoop.io.DataOutputBuffer buffer) Adds the row to the batch after deserializing the row
static ColumnVector	allocateColumnVector(String type, int defaultSize)
void	convertRowBatchBlobToVectorizedBatch(Object rowBlob, int rowsInBlob, VectorizedRowBatch batch) Deserialized set of rows and populates the batch
VectorizedRowBatch	createVectorizedRowBatch() Creates a Vectorized row batch and the column vectors.
void	init(org.apache.hadoop.conf.Configuration hiveConf, org.apache.hadoop.mapred.FileSplit split) Initializes VectorizedRowBatch context based on the split and Hive configuration (Job conf with hive Plan).
void	init(Map<Integer,String> scratchColumnTypeMap, StructObjectInspector rowOI) Initializes the VectorizedRowBatch context based on an scratch column type map and object inspector.
boolean	isPartitionCol(int colnum) Determine whether a given column is a partition column

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorizedRowBatchCtx

public VectorizedRowBatchCtx(StructObjectInspector rawRowOI,
                     StructObjectInspector rowOI,
                     Deserializer deserializer,
                     Map<String,Object> partitionValues,
                     Map<String,PrimitiveObjectInspector.PrimitiveCategory> partitionTypes)

Constructor for VectorizedRowBatchCtx

Parameters:

rawRowOI - OI for raw row data (EG without partition cols)

rowOI - OI for the row (Raw row OI + partition OI)

deserializer - Deserializer for the row data

partitionValues - Hash map of partition values. Key=TblColName value=PartitionValue

VectorizedRowBatchCtx

public VectorizedRowBatchCtx()

Constructor for VectorizedRowBatchCtx

Method Detail

init

public void init(Map<Integer,String> scratchColumnTypeMap,
        StructObjectInspector rowOI)

Initializes the VectorizedRowBatch context based on an scratch column type map and object inspector.

Parameters:

scratchColumnTypeMap -

rowOI - Object inspector that shapes the column types

init

public void init(org.apache.hadoop.conf.Configuration hiveConf,
        org.apache.hadoop.mapred.FileSplit split)
          throws ClassNotFoundException,
                 IOException,
                 SerDeException,
                 InstantiationException,
                 IllegalAccessException,
                 HiveException

Initializes VectorizedRowBatch context based on the split and Hive configuration (Job conf with hive Plan).

Parameters:

hiveConf - Hive configuration using Hive plan is extracted

split - File split of the file being read

Throws:

ClassNotFoundException

IOException

SerDeException

InstantiationException

IllegalAccessException

HiveException

createVectorizedRowBatch

public VectorizedRowBatch createVectorizedRowBatch()
                                            throws HiveException

Creates a Vectorized row batch and the column vectors.

Returns:

VectorizedRowBatch

Throws:

HiveException

addRowToBatch

public void addRowToBatch(int rowIndex,
                 org.apache.hadoop.io.Writable rowBlob,
                 VectorizedRowBatch batch,
                 org.apache.hadoop.io.DataOutputBuffer buffer)
                   throws HiveException,
                          SerDeException

Adds the row to the batch after deserializing the row

Parameters:

rowIndex - Row index in the batch to which the row is added

rowBlob - Row blob (serialized version of row)

batch - Vectorized batch to which the row is added

buffer - a buffer to copy strings into

Throws:

HiveException

SerDeException

convertRowBatchBlobToVectorizedBatch

public void convertRowBatchBlobToVectorizedBatch(Object rowBlob,
                                        int rowsInBlob,
                                        VectorizedRowBatch batch)
                                          throws SerDeException

Deserialized set of rows and populates the batch

Parameters:

rowBlob - to deserialize

batch - Vectorized row batch which contains deserialized data

Throws:

SerDeException

addPartitionColsToBatch

public void addPartitionColsToBatch(VectorizedRowBatch batch)
                             throws HiveException

Add the partition values to the batch

Parameters:

batch -

Throws:

HiveException

isPartitionCol

public final boolean isPartitionCol(int colnum)

Determine whether a given column is a partition column

Parameters:

colnum - column number in VectorizedRowBatchs created by this context.

Returns:

true if it is a partition column, false otherwise

allocateColumnVector

public static ColumnVector allocateColumnVector(String type,
                                int defaultSize)