org.apache.hadoop.hive.ql.plan

Class ReduceWork

java.lang.Object

org.apache.hadoop.hive.ql.plan.AbstractOperatorDesc

org.apache.hadoop.hive.ql.plan.BaseWork

org.apache.hadoop.hive.ql.plan.ReduceWork

All Implemented Interfaces:

Serializable, Cloneable, OperatorDesc

public class ReduceWork
extends BaseWork

ReduceWork represents all the information used to run a reduce task on the cluster. It is first used when the query planner breaks the logical plan into tasks and used throughout physical optimization to track reduce-side operator plans, schema info about key/value pairs, etc ExecDriver will serialize the contents of this class and make sure it is distributed on the cluster. The ExecReducer will ultimately deserialize this class on the data nodes and setup it's operator pipeline accordingly. This class is also used in the explain command any property with the appropriate annotation will be displayed in the explain output.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.plan.BaseWork

vectorColumnNameMap, vectorColumnTypeMap, vectorScratchColumnTypeMap

Fields inherited from class org.apache.hadoop.hive.ql.plan.AbstractOperatorDesc

opProps, opTraits, statistics, vectorMode

Constructor Summary

Constructors

Constructor and Description
ReduceWork()
ReduceWork(String name)

Method Summary

Methods

Modifier and Type	Method and Description
void	configureJobConf(org.apache.hadoop.mapred.JobConf job)
Set<Operator<?>>	getAllRootOperators()
TableDesc	getKeyDesc()
ObjectInspector	getKeyObjectInspector()
int	getMaxReduceTasks()
int	getMinReduceTasks()
boolean	getNeedsTagging()
Integer	getNumReduceTasks() If the number of reducers is -1, the runtime will automatically figure it out by input data size.
Operator<?>	getReducer()
Map<Integer,String>	getTagToInput()
List<TableDesc>	getTagToValueDesc()
ObjectInspector	getValueObjectInspector()
String	getVectorModeOn()
boolean	isAutoReduceParallelism()
void	replaceRoots(Map<Operator<?>,Operator<?>> replacementMap)
void	setAutoReduceParallelism(boolean isAutoReduceParallelism)
void	setKeyDesc(TableDesc keyDesc) If the plan has a reducer and correspondingly a reduce-sink, then store the TableDesc pointing to keySerializeInfo of the ReduceSink
void	setMaxReduceTasks(int maxReduceTasks)
void	setMinReduceTasks(int minReduceTasks)
void	setNeedsTagging(boolean needsTagging)
void	setNumReduceTasks(Integer numReduceTasks)
void	setReducer(Operator<?> reducer)
void	setTagToInput(Map<Integer,String> tagToInput)
void	setTagToValueDesc(List<TableDesc> tagToValueDesc)

Methods inherited from class org.apache.hadoop.hive.ql.plan.BaseWork

addDummyOp, addSortCols, getAllLeafOperators, getAllOperators, getDummyOps, getMapRedLocalWork, getName, getSortCols, getTag, getVectorColumnNameMap, getVectorColumnTypeMap, getVectorScratchColumnTypeMap, isGatheringStats, setDummyOps, setGatheringStats, setMapRedLocalWork, setName, setTag, setVectorColumnNameMap, setVectorColumnTypeMap, setVectorScratchColumnTypeMap

Methods inherited from class org.apache.hadoop.hive.ql.plan.AbstractOperatorDesc

clone, getOpProps, getStatistics, getTraits, getVectorMode, setOpProps, setStatistics, setTraits, setVectorMode

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReduceWork

public ReduceWork()

ReduceWork

public ReduceWork(String name)

Method Detail

setKeyDesc

public void setKeyDesc(TableDesc keyDesc)

If the plan has a reducer and correspondingly a reduce-sink, then store the TableDesc pointing to keySerializeInfo of the ReduceSink

Parameters:

keyDesc -

getKeyDesc

public TableDesc getKeyDesc()

getKeyObjectInspector

public ObjectInspector getKeyObjectInspector()

getValueObjectInspector

public ObjectInspector getValueObjectInspector()

getTagToValueDesc

public List<TableDesc> getTagToValueDesc()

setTagToValueDesc

public void setTagToValueDesc(List<TableDesc> tagToValueDesc)

getVectorModeOn

public String getVectorModeOn()

getReducer

public Operator<?> getReducer()

setReducer

public void setReducer(Operator<?> reducer)

getNeedsTagging

public boolean getNeedsTagging()

setNeedsTagging

public void setNeedsTagging(boolean needsTagging)

setTagToInput

public void setTagToInput(Map<Integer,String> tagToInput)

getTagToInput

public Map<Integer,String> getTagToInput()

replaceRoots

public void replaceRoots(Map<Operator<?>,Operator<?>> replacementMap)

Specified by:

replaceRoots in class BaseWork

getAllRootOperators

public Set<Operator<?>> getAllRootOperators()

Specified by:

getAllRootOperators in class BaseWork

getNumReduceTasks

public Integer getNumReduceTasks()

If the number of reducers is -1, the runtime will automatically figure it out by input data size. The number of reducers will be a positive number only in case the target table is bucketed into N buckets (through CREATE TABLE). This feature is not supported yet, so the number of reducers will always be -1 for now.

setNumReduceTasks

public void setNumReduceTasks(Integer numReduceTasks)

configureJobConf

public void configureJobConf(org.apache.hadoop.mapred.JobConf job)

Specified by:

configureJobConf in class BaseWork

setAutoReduceParallelism

public void setAutoReduceParallelism(boolean isAutoReduceParallelism)

isAutoReduceParallelism

public boolean isAutoReduceParallelism()

setMinReduceTasks

public void setMinReduceTasks(int minReduceTasks)

getMinReduceTasks

public int getMinReduceTasks()

getMaxReduceTasks

public int getMaxReduceTasks()

setMaxReduceTasks

public void setMaxReduceTasks(int maxReduceTasks)