org.apache.hadoop.hive.ql.exec

Class PTFTopNHash

java.lang.Object

org.apache.hadoop.hive.ql.exec.TopNHash

org.apache.hadoop.hive.ql.exec.PTFTopNHash

public class PTFTopNHash
extends TopNHash

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.TopNHash

TopNHash.BinaryCollector

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	isMapGroupBy
protected float	memUsage

Fields inherited from class org.apache.hadoop.hive.ql.exec.TopNHash

batchIndexToResult, batchSize, collector, EXCLUDE, FORWARD, isEnabled, LOG, threshold, topN, usage

Constructor Summary

Constructors

Constructor and Description
PTFTopNHash()

Method Summary

Methods

Modifier and Type	Method and Description
int	_tryStoreKey(HiveKey key, boolean partColsIsNull, int batchIndex)
void	flush() Flushes all the rows cached in the heap.
int	getVectorizedBatchResult(int batchIndex) Get vectorized batch result for particular index.
int	getVectorizedKeyDistLength(int batchIndex) After vectorized batch is processed, can return distribution keys length of a key.
int	getVectorizedKeyHashCode(int batchIndex) After vectorized batch is processed, can return hashCode of a key.
HiveKey	getVectorizedKeyToForward(int batchIndex) After vectorized batch is processed, can return the key that caused a particular row to be forwarded.
void	initialize(int topN, float memUsage, boolean isMapGroupBy, TopNHash.BinaryCollector collector)
int	startVectorizedBatch(int size) Perform basic checks and initialize TopNHash for the new vectorized row batch.
void	storeValue(int index, int hashCode, org.apache.hadoop.io.BytesWritable value, boolean vectorized) Stores the value for the key in the heap.
int	tryStoreKey(HiveKey key, boolean partColsIsNull) Try store the non-vectorized key.
void	tryStoreVectorizedKey(HiveKey key, boolean partColsIsNull, int batchIndex) Try to put the key from the current vectorized batch into the heap.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

memUsage

protected float memUsage

isMapGroupBy

protected boolean isMapGroupBy

Constructor Detail

PTFTopNHash

public PTFTopNHash()

Method Detail

initialize

public void initialize(int topN,
              float memUsage,
              boolean isMapGroupBy,
              TopNHash.BinaryCollector collector)

Overrides:

initialize in class TopNHash

tryStoreKey

public int tryStoreKey(HiveKey key,
              boolean partColsIsNull)
                throws HiveException,
                       IOException

Description copied from class: TopNHash

Try store the non-vectorized key.

Overrides:

tryStoreKey in class TopNHash

Parameters:

key - Serialized key.

Returns:

TopNHash.FORWARD if the row should be forwarded; TopNHash.EXCLUDED if the row should be discarded; any other number if the row is to be stored; the index should be passed to storeValue.

Throws:

HiveException

IOException

_tryStoreKey

public int _tryStoreKey(HiveKey key,
               boolean partColsIsNull,
               int batchIndex)
                 throws HiveException,
                        IOException

Throws:

HiveException

IOException

storeValue

public void storeValue(int index,
              int hashCode,
              org.apache.hadoop.io.BytesWritable value,
              boolean vectorized)

Description copied from class: TopNHash

Stores the value for the key in the heap.

Overrides:

storeValue in class TopNHash

Parameters:

index - The index, either from tryStoreKey or from tryStoreVectorizedKey result.

value - The value to store.

vectorized - Whether the result is coming from a vectorized batch.

flush

public void flush()
           throws HiveException

Description copied from class: TopNHash

Flushes all the rows cached in the heap.

Overrides:

flush in class TopNHash

Throws:

HiveException

startVectorizedBatch

public int startVectorizedBatch(int size)
                         throws IOException,
                                HiveException

Description copied from class: TopNHash

Perform basic checks and initialize TopNHash for the new vectorized row batch.

Overrides:

startVectorizedBatch in class TopNHash

Parameters:

size - batch size

Returns:

TopNHash.FORWARD if all rows should be forwarded w/o trying to call TopN; TopNHash.EXCLUDED if all rows should be discarded w/o trying to call TopN; any other result means the batch has been started.

Throws:

IOException

HiveException

tryStoreVectorizedKey

public void tryStoreVectorizedKey(HiveKey key,
                         boolean partColsIsNull,
                         int batchIndex)
                           throws HiveException,
                                  IOException

Description copied from class: TopNHash

Try to put the key from the current vectorized batch into the heap.

Overrides:

tryStoreVectorizedKey in class TopNHash

Parameters:

key - the key.

batchIndex - The index of the key in the vectorized batch (sequential, not .selected).

Throws:

HiveException

IOException

getVectorizedBatchResult

public int getVectorizedBatchResult(int batchIndex)

Description copied from class: TopNHash

Get vectorized batch result for particular index.

Overrides:

getVectorizedBatchResult in class TopNHash

Parameters:

batchIndex - index of the key in the batch.

Returns:

the result, same as from #tryStoreKey(HiveKey)

getVectorizedKeyToForward

public HiveKey getVectorizedKeyToForward(int batchIndex)

Description copied from class: TopNHash

After vectorized batch is processed, can return the key that caused a particular row to be forwarded. Because the row could only be marked to forward because it has the same key with some row already in the heap (for GBY), we can use that key from the heap to emit the forwarded row.

Overrides:

getVectorizedKeyToForward in class TopNHash

Parameters:

batchIndex - index of the key in the batch.

Returns:

The key corresponding to the index.

getVectorizedKeyDistLength

public int getVectorizedKeyDistLength(int batchIndex)

Description copied from class: TopNHash

After vectorized batch is processed, can return distribution keys length of a key.

Overrides:

getVectorizedKeyDistLength in class TopNHash

Parameters:

batchIndex - index of the key in the batch.

Returns:

The distribution length corresponding to the key.

getVectorizedKeyHashCode

public int getVectorizedKeyHashCode(int batchIndex)

Description copied from class: TopNHash

After vectorized batch is processed, can return hashCode of a key.

Overrides:

getVectorizedKeyHashCode in class TopNHash

Parameters:

batchIndex - index of the key in the batch.

Returns:

The hashCode corresponding to the key.