org.apache.hadoop.hive.ql.exec.persistence

Class HashMapWrapper

java.lang.Object

org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper

All Implemented Interfaces:

Serializable, MapJoinPersistableTableContainer, MapJoinTableContainer

public class HashMapWrapper
extends AbstractMapJoinTableContainer
implements Serializable

Simple wrapper for persistent Hashmap implementing only the put/get/remove/clear interface. The main memory hash table acts as a cache and all put/get will operate on it first. If the size of the main memory hash table exceeds a certain threshold, new elements will go into the persistent hash table.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hive.ql.exec.persistence.MapJoinTableContainer

MapJoinTableContainer.ReusableGetAdaptor

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.commons.logging.Log	LOG

Fields inherited from class org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

LOAD_NAME, THESHOLD_NAME

Constructor Summary

Constructors

Constructor and Description
HashMapWrapper()
HashMapWrapper(org.apache.hadoop.conf.Configuration hconf, long keyCount)
HashMapWrapper(Map<String,String> metaData)

Method Summary

Methods

Modifier and Type	Method and Description
static int	calculateTableSize(float keyCountAdj, int threshold, float loadFactor, long keyCount)
void	clear() Clears the contents of the table.
MapJoinTableContainer.ReusableGetAdaptor	createGetter(MapJoinKey keyTypeFromLoader) Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.
void	dumpMetrics()
Set<Map.Entry<MapJoinKey,MapJoinRowContainer>>	entrySet()
MapJoinRowContainer	get(MapJoinKey key)
MapJoinKey	getAnyKey()
boolean	hasSpill() Checks if the container has spilled any data onto disk.
void	put(MapJoinKey key, MapJoinRowContainer value)
MapJoinKey	putRow(MapJoinObjectSerDeContext keyContext, org.apache.hadoop.io.Writable currentKey, MapJoinObjectSerDeContext valueContext, org.apache.hadoop.io.Writable currentValue) Adds row from input to the table.
void	seal() Indicates to the container that the puts have ended; table is now r/o.
int	size()

Methods inherited from class org.apache.hadoop.hive.ql.exec.persistence.AbstractMapJoinTableContainer

createConstructorMetaData, getMetaData, putMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.apache.commons.logging.Log LOG

Constructor Detail

HashMapWrapper

public HashMapWrapper(Map<String,String> metaData)

HashMapWrapper

public HashMapWrapper()

HashMapWrapper

public HashMapWrapper(org.apache.hadoop.conf.Configuration hconf,
              long keyCount)

Method Detail

calculateTableSize

public static int calculateTableSize(float keyCountAdj,
                     int threshold,
                     float loadFactor,
                     long keyCount)

get

public MapJoinRowContainer get(MapJoinKey key)

Specified by:

get in interface MapJoinPersistableTableContainer

put

public void put(MapJoinKey key,
       MapJoinRowContainer value)

Specified by:

put in interface MapJoinPersistableTableContainer

size

public int size()

Specified by:

size in interface MapJoinPersistableTableContainer

entrySet

public Set<Map.Entry<MapJoinKey,MapJoinRowContainer>> entrySet()

Specified by:

entrySet in interface MapJoinPersistableTableContainer

clear

public void clear()

Description copied from interface: MapJoinTableContainer

Clears the contents of the table.

Specified by:

clear in interface MapJoinTableContainer

putRow

public MapJoinKey putRow(MapJoinObjectSerDeContext keyContext,
                org.apache.hadoop.io.Writable currentKey,
                MapJoinObjectSerDeContext valueContext,
                org.apache.hadoop.io.Writable currentValue)
                  throws SerDeException,
                         HiveException

Description copied from interface: MapJoinTableContainer

Adds row from input to the table.

Specified by:

putRow in interface MapJoinTableContainer

Throws:

SerDeException

HiveException

createGetter

public MapJoinTableContainer.ReusableGetAdaptor createGetter(MapJoinKey keyTypeFromLoader)

Description copied from interface: MapJoinTableContainer

Creates reusable get adaptor that can be used to retrieve rows from the table based on either vectorized or non-vectorized input rows to MapJoinOperator.

Specified by:

createGetter in interface MapJoinTableContainer

Parameters:

keyTypeFromLoader - Last key from hash table loader, to determine key type used when loading hashtable (if it can vary).

seal

public void seal()

Description copied from interface: MapJoinTableContainer

Indicates to the container that the puts have ended; table is now r/o.

Specified by:

seal in interface MapJoinTableContainer

getAnyKey

public MapJoinKey getAnyKey()

Specified by:

getAnyKey in interface MapJoinTableContainer

dumpMetrics

public void dumpMetrics()

Specified by:

dumpMetrics in interface MapJoinTableContainer

hasSpill

public boolean hasSpill()

Description copied from interface: MapJoinTableContainer

Checks if the container has spilled any data onto disk. This is only applicable for HybridHashTableContainer.

Specified by:

hasSpill in interface MapJoinTableContainer