org.apache.hadoop.hive.ql.plan

Class ListBucketingCtx

java.lang.Object

org.apache.hadoop.hive.ql.plan.ListBucketingCtx

All Implemented Interfaces:

Serializable

public class ListBucketingCtx
extends Object
implements Serializable

Context for list bucketing. It's created in SemanticAnalyzer.genFileSinkPlan(). It's used in FileSinkOperator.processOp(), merging files, alter table ...concatenate etc.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ListBucketingCtx()

Method Summary

Methods

Modifier and Type	Method and Description
int	calculateListBucketingLevel() Calculate list bucketing level.
void	calculateSkewedValueSubDirList() Calculate skewed value subdirectory directory which is used in FileSinkOperator.java createKeyForStatsPublisher() For example, create table test skewed by (key, value) on (('484','val_484') stored as DIRECTORIES; after the method, skewedValuesDirNames will contain 2 elements: key=484/value=val_484 HIVE_LIST_BUCKETING_DEFAULT_DIR_NAME/HIVE_LIST_BUCKETING_DEFAULT_DIR_NAME
String	getDefaultDirName()
String	getDefaultKey()
Map<List<String>,String>	getLbLocationMap()
List<SkewedColumnPositionPair>	getRowSkewedIndex()
List<String>	getSkewedColNames()
List<List<String>>	getSkewedColValues()
List<String>	getSkewedValuesDirNames()
boolean	isSkewedStoredAsDir() check if list bucketing is enabled.
boolean	isStoredAsSubDirectories()
void	processRowSkewedIndex(RowSchema rowSch) Match column in skewed column list and record position.
void	setDefaultDirName(String defaultDirName)
void	setDefaultKey(String defaultKey)
void	setLbLocationMap(Map<List<String>,String> lbLocationMap)
void	setRowSkewedIndex(List<SkewedColumnPositionPair> rowSkewedIndex)
void	setSkewedColNames(List<String> skewedColNames)
void	setSkewedColValues(List<List<String>> skewedColValues)
void	setSkewedValuesDirNames(List<String> skewedValuesDirNames)
void	setStoredAsSubDirectories(boolean isStoredAsSubDirectories)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ListBucketingCtx

public ListBucketingCtx()

Method Detail

getSkewedColNames

public List<String> getSkewedColNames()

Returns:

the skewedColNames

setSkewedColNames

public void setSkewedColNames(List<String> skewedColNames)

Parameters:

skewedColNames - the skewedColNames to set

getSkewedColValues

public List<List<String>> getSkewedColValues()

Returns:

the skewedColValues

setSkewedColValues

public void setSkewedColValues(List<List<String>> skewedColValues)

Parameters:

skewedColValues - the skewedColValues to set

getLbLocationMap

public Map<List<String>,String> getLbLocationMap()

Returns:

the lbLocationMap

setLbLocationMap

public void setLbLocationMap(Map<List<String>,String> lbLocationMap)

Parameters:

lbLocationMap - the lbLocationMap to set

processRowSkewedIndex

public void processRowSkewedIndex(RowSchema rowSch)

Match column in skewed column list and record position. The position will be used in FileSinkOperator generateListBucketingDirName(). Note that skewed column name matches skewed value in order.

Parameters:

rowSch -

calculateSkewedValueSubDirList

public void calculateSkewedValueSubDirList()

Calculate skewed value subdirectory directory which is used in FileSinkOperator.java createKeyForStatsPublisher() For example, create table test skewed by (key, value) on (('484','val_484') stored as DIRECTORIES; after the method, skewedValuesDirNames will contain 2 elements: key=484/value=val_484 HIVE_LIST_BUCKETING_DEFAULT_DIR_NAME/HIVE_LIST_BUCKETING_DEFAULT_DIR_NAME

isStoredAsSubDirectories

public boolean isStoredAsSubDirectories()

Returns:

the isStoredAsSubDirectories

setStoredAsSubDirectories

public void setStoredAsSubDirectories(boolean isStoredAsSubDirectories)

Parameters:

isStoredAsSubDirectories - the isStoredAsSubDirectories to set

getDefaultKey

public String getDefaultKey()

Returns:

the defaultKey

setDefaultKey

public void setDefaultKey(String defaultKey)

Parameters:

defaultKey - the defaultKey to set

getDefaultDirName

public String getDefaultDirName()

Returns:

the defaultDirName

setDefaultDirName

public void setDefaultDirName(String defaultDirName)

Parameters:

defaultDirName - the defaultDirName to set

isSkewedStoredAsDir

public boolean isSkewedStoredAsDir()

check if list bucketing is enabled.

Parameters:

ctx -

Returns:

calculateListBucketingLevel

public int calculateListBucketingLevel()

Calculate list bucketing level. 0: not list bucketing int: no. of skewed columns

Parameters:

ctx -

Returns:

getSkewedValuesDirNames

public List<String> getSkewedValuesDirNames()

Returns:

the skewedValuesDirNames

setSkewedValuesDirNames

public void setSkewedValuesDirNames(List<String> skewedValuesDirNames)

Parameters:

skewedValuesDirNames - the skewedValuesDirNames to set

getRowSkewedIndex

public List<SkewedColumnPositionPair> getRowSkewedIndex()

Returns:

the rowSkewedIndex

setRowSkewedIndex

public void setRowSkewedIndex(List<SkewedColumnPositionPair> rowSkewedIndex)

Parameters:

rowSkewedIndex - the rowSkewedIndex to set