Modifier and Type | Class and Description |
---|---|
class |
AbstractFileMergeOperator<T extends FileMergeDesc>
Fast file merge operator for ORC and RCfile.
|
class |
AbstractMapJoinOperator<T extends MapJoinDesc> |
class |
AbstractMapOperator
Abstract Map operator.
|
class |
AppMasterEventOperator
AppMasterEventOperator sends any rows it receives to the Tez AM.
|
class |
CollectOperator
Buffers rows emitted by other operators.
|
class |
CommonJoinOperator<T extends JoinDesc>
Join operator implementation.
|
class |
CommonMergeJoinOperator |
class |
DemuxOperator
DemuxOperator is an operator used by MapReduce Jobs optimized by
CorrelationOptimizer.
|
class |
DummyStoreOperator
For SortMerge joins, this is a dummy operator, which stores the row for the
small table before it reaches the sort merge join operator.
|
class |
FileSinkOperator
File Sink operator implementation.
|
class |
FilterOperator
Filter operator implementation.
|
class |
ForwardOperator
Forward Operator Just forwards.
|
class |
GroupByOperator
GroupBy operator implementation.
|
class |
HashTableDummyOperator |
class |
HashTableSinkOperator |
class |
JoinOperator
Join operator implementation.
|
class |
LateralViewForwardOperator
LateralViewForwardOperator.
|
class |
LateralViewJoinOperator
The lateral view join operator is used for FROM src LATERAL VIEW udtf()...
|
class |
LimitOperator
Limit operator implementation Limits the number of rows to be passed on.
|
class |
ListSinkOperator
For fetch task with operator tree, row read from FetchOperator is processed via operator tree
and finally arrives to this operator.
|
class |
MapJoinOperator
Map side Join operator implementation.
|
class |
MapOperator
Map operator.
|
class |
MuxOperator
MuxOperator is used in the Reduce side of MapReduce jobs optimized by Correlation Optimizer.
|
class |
OrcFileMergeOperator
Fast file merge operator for ORC files.
|
class |
PTFOperator |
class |
RCFileMergeOperator
Fast file merge operator for RC files.
|
class |
ReduceSinkOperator
Reduce Sink Operator sends output to the reduce stage.
|
class |
ScriptOperator
ScriptOperator.
|
class |
SelectOperator
Select operator implementation.
|
class |
SMBMapJoinOperator
Sorted Merge Map Join Operator.
|
class |
SparkHashTableSinkOperator |
class |
TableScanOperator
Table Scan Operator If the data is coming from the map-reduce framework, just
forward it.
|
class |
TemporaryHashSinkOperator |
class |
TerminalOperator<T extends OperatorDesc>
Terminal Operator Base Class.
|
class |
TezDummyStoreOperator
A dummy store operator same as the dummy store operator but for tez.
|
class |
UDTFOperator
UDTFOperator.
|
class |
UnionOperator
Union Operator Just forwards.
|
Modifier and Type | Field and Description |
---|---|
protected Operator<? extends OperatorDesc>[] |
Operator.childOperatorsArray
Cache childOperators in an array for faster access.
|
Modifier and Type | Field and Description |
---|---|
protected List<Operator<? extends OperatorDesc>> |
Operator.childOperators |
protected List<Operator<? extends OperatorDesc>> |
Operator.parentOperators |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
TableScanOperator.clone() |
Operator<? extends OperatorDesc> |
Operator.clone() |
Operator<? extends OperatorDesc> |
Operator.cloneOp()
Clones only the operator.
|
Operator<? extends OperatorDesc> |
Operator.cloneRecursiveChildren()
Recursively clones all the children of the tree,
Fixes the pointers to children, parents and the pointers to itself coming from the children.
|
static Operator |
Operator.createDummy() |
static Operator<?> |
OperatorUtils.findOperatorByMarker(Operator<?> start,
String marker) |
static <T extends OperatorDesc> |
OperatorFactory.get(CompilationOpContext cContext,
Class<T> descClass) |
static <T extends OperatorDesc> |
OperatorFactory.get(CompilationOpContext cContext,
T conf)
Returns an operator given the conf and a list of children operators.
|
static <T extends OperatorDesc> |
OperatorFactory.get(CompilationOpContext cContext,
T conf,
RowSchema rwsch)
Returns an operator given the conf and a list of children operators.
|
static <T extends OperatorDesc> |
OperatorFactory.get(T conf,
Operator<? extends OperatorDesc> oplist0,
Operator<? extends OperatorDesc>... oplist)
Returns an operator given the conf and a list of children operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
RowSchema rwsch)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
RowSchema rwsch,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
RowSchema rwsch,
Map<String,ExprNodeDesc> colExprMap,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext ctx,
T conf,
RowSchema rwsch,
Operator[] oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Map<String,ExprNodeDesc> colExprMap,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
Operator<? extends OperatorDesc> |
Task.getReducer(MapWork work) |
static <T extends OperatorDesc> |
OperatorFactory.getVectorOperator(Class<? extends Operator<?>> opClass,
CompilationOpContext cContext,
T conf,
VectorizationContext vContext,
VectorDesc vectorDesc) |
static <T extends OperatorDesc> |
OperatorFactory.getVectorOperator(CompilationOpContext cContext,
T conf,
VectorizationContext vContext,
VectorDesc vectorDesc) |
Modifier and Type | Method and Description |
---|---|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperators(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
in the stream depending on the classes they instantiate.
|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperators(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
in the stream depending on the classes they instantiate.
|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
upstream depending on the classes it instantiates.
|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
upstream depending on the classes it instantiates.
|
static List<Operator<?>> |
SerializationUtilities.cloneOperatorTree(List<Operator<?>> roots)
Clones using the powers of XML.
|
static List<Operator<?>> |
SerializationUtilities.cloneOperatorTree(List<Operator<?>> roots,
int indexForTezUnion) |
List<Operator<? extends OperatorDesc>> |
Operator.getChildOperators() |
static Set<Operator<?>> |
OperatorUtils.getOp(BaseWork work,
Class<?> clazz) |
List<Operator<? extends OperatorDesc>> |
Operator.getParentOperators() |
Collection<Operator<? extends OperatorDesc>> |
Task.getTopOperators() |
Modifier and Type | Method and Description |
---|---|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperators(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
in the stream depending on the classes they instantiate.
|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
upstream depending on the classes it instantiates.
|
static int |
OperatorUtils.countOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it returns the number of operators it finds
upstream that instantiate any of the given classes.
|
static FetchOperator |
PartitionKeySampler.createSampler(FetchWork work,
org.apache.hadoop.mapred.JobConf job,
Operator<?> operator) |
static <T> T |
OperatorUtils.findLastOperator(Operator<?> op,
Class<T> clazz)
Starting at the input operator, finds the last operator in the stream that
is an instance of the input class.
|
static Operator<?> |
OperatorUtils.findOperatorByMarker(Operator<?> start,
String marker) |
static <T> Set<T> |
OperatorUtils.findOperators(Operator<?> start,
Class<T> clazz) |
static <T> Set<T> |
OperatorUtils.findOperatorsUpstream(Operator<?> start,
Class<T> clazz) |
static <T> Set<T> |
OperatorUtils.findOperatorsUpstreamJoinAccounted(Operator<?> start,
Class<T> clazz,
Set<T> found) |
static void |
OperatorUtils.findRoots(Operator<?> op,
Collection<Operator<?>> roots)
Given the input operator 'op', walk up the operator tree from 'op', and collect all the
roots that can be reached from it.
|
static <T> T |
OperatorUtils.findSingleOperator(Operator<?> start,
Class<T> clazz) |
static <T> T |
OperatorUtils.findSingleOperatorUpstream(Operator<?> start,
Class<T> clazz) |
static <T> T |
OperatorUtils.findSingleOperatorUpstreamJoinAccounted(Operator<?> start,
Class<T> clazz) |
void |
Operator.OperatorFunc.func(Operator<? extends OperatorDesc> op) |
static <T extends OperatorDesc> |
OperatorFactory.get(T conf,
Operator<? extends OperatorDesc> oplist0,
Operator<? extends OperatorDesc>... oplist)
Returns an operator given the conf and a list of children operators.
|
static <T extends OperatorDesc> |
OperatorFactory.get(T conf,
Operator<? extends OperatorDesc> oplist0,
Operator<? extends OperatorDesc>... oplist)
Returns an operator given the conf and a list of children operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext ctx,
T conf,
RowSchema rwsch,
Operator[] oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Map<String,ExprNodeDesc> colExprMap,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Map<String,ExprNodeDesc> colExprMap,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(T conf,
RowSchema rwsch,
Operator oplist0,
Operator... oplist)
Returns an operator given the conf and a list of parent operators.
|
static String |
OperatorUtils.getOpNamePretty(Operator<?> op) |
static String |
Utilities.getOpTreeSkel(Operator<?> op) |
int |
CommonMergeJoinOperator.getTagForOperator(Operator<? extends OperatorDesc> op) |
static void |
OperatorUtils.iterateParents(Operator<?> operator,
NodeUtils.Function<Operator<?>> function) |
boolean |
ForwardOperator.logicalEquals(Operator other) |
boolean |
DemuxOperator.logicalEquals(Operator other) |
boolean |
MuxOperator.logicalEquals(Operator other) |
boolean |
LateralViewForwardOperator.logicalEquals(Operator other) |
boolean |
Operator.logicalEquals(Operator other)
Decides whether two operators are logically the same.
|
boolean |
ListSinkOperator.logicalEquals(Operator other) |
boolean |
UnionOperator.logicalEquals(Operator other) |
boolean |
CollectOperator.logicalEquals(Operator other) |
boolean |
Operator.logicalEqualsTree(Operator<?> o)
Compares the whole operator tree with the other.
|
static void |
OperatorFactory.makeChild(Operator<? extends OperatorDesc> ret,
Operator<? extends OperatorDesc>... oplist)
Returns an operator given the conf and a list of children operators.
|
static void |
OperatorFactory.makeChild(Operator<? extends OperatorDesc> ret,
Operator<? extends OperatorDesc>... oplist)
Returns an operator given the conf and a list of children operators.
|
void |
Operator.removeChild(Operator<? extends OperatorDesc> child) |
void |
Operator.removeChildAndAdoptItsChildren(Operator<? extends OperatorDesc> child)
Remove a child and add all of the child's children to the location of the child
|
static void |
OperatorUtils.removeOperator(Operator<?> op)
Remove operator from the tree, disconnecting it from its
parents and children.
|
void |
Operator.removeParent(Operator<? extends OperatorDesc> parent) |
void |
Operator.replaceChild(Operator<? extends OperatorDesc> child,
Operator<? extends OperatorDesc> newChild)
Replace one child with another at the same position.
|
void |
Operator.replaceChild(Operator<? extends OperatorDesc> child,
Operator<? extends OperatorDesc> newChild)
Replace one child with another at the same position.
|
void |
Operator.replaceParent(Operator<? extends OperatorDesc> parent,
Operator<? extends OperatorDesc> newParent)
Replace one parent with another at the same position.
|
void |
Operator.replaceParent(Operator<? extends OperatorDesc> parent,
Operator<? extends OperatorDesc> newParent)
Replace one parent with another at the same position.
|
static void |
Utilities.setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
Operator op) |
static void |
Utilities.setColumnNameList(org.apache.hadoop.mapred.JobConf jobConf,
Operator op,
boolean excludeVCs) |
static void |
Utilities.setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
Operator op) |
static void |
Utilities.setColumnTypeList(org.apache.hadoop.mapred.JobConf jobConf,
Operator op,
boolean excludeVCs) |
Modifier and Type | Method and Description |
---|---|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperators(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
in the stream depending on the classes they instantiate.
|
static com.google.common.collect.Multimap<Class<? extends Operator<?>>,Operator<?>> |
OperatorUtils.classifyOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it classifies the operators it finds
upstream depending on the classes it instantiates.
|
static List<Operator<?>> |
SerializationUtilities.cloneOperatorTree(List<Operator<?>> roots)
Clones using the powers of XML.
|
static List<Operator<?>> |
SerializationUtilities.cloneOperatorTree(List<Operator<?>> roots,
int indexForTezUnion) |
void |
CommonMergeJoinOperator.cloneOriginalParentsList(List<Operator<? extends OperatorDesc>> opList) |
static int |
OperatorUtils.countOperatorsUpstream(Operator<?> start,
Set<Class<? extends Operator<?>>> classes)
Given an operator and a set of classes, it returns the number of operators it finds
upstream that instantiate any of the given classes.
|
static <T> Set<T> |
OperatorUtils.findOperators(Collection<Operator<?>> starts,
Class<T> clazz) |
static <T> Set<T> |
OperatorUtils.findOperatorsUpstream(Collection<Operator<?>> starts,
Class<T> clazz) |
static void |
OperatorUtils.findRoots(Operator<?> op,
Collection<Operator<?>> roots)
Given the input operator 'op', walk up the operator tree from 'op', and collect all the
roots that can be reached from it.
|
List<String> |
GroupByOperator.genColLists(HashMap<Operator<? extends OperatorDesc>,OpParseContext> opParseCtx) |
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
RowSchema rwsch,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getAndMakeChild(CompilationOpContext cContext,
T conf,
RowSchema rwsch,
Map<String,ExprNodeDesc> colExprMap,
List<Operator<? extends OperatorDesc>> oplist)
Returns an operator given the conf and a list of parent operators.
|
static <T extends OperatorDesc> |
OperatorFactory.getVectorOperator(Class<? extends Operator<?>> opClass,
CompilationOpContext cContext,
T conf,
VectorizationContext vContext,
VectorDesc vectorDesc) |
void |
MapOperator.initEmptyInputChildren(List<Operator<?>> children,
org.apache.hadoop.conf.Configuration hconf) |
abstract void |
AbstractMapOperator.initEmptyInputChildren(List<Operator<?>> children,
org.apache.hadoop.conf.Configuration hconf) |
static void |
OperatorUtils.iterateParents(Operator<?> operator,
NodeUtils.Function<Operator<?>> function) |
void |
Operator.setChildOperators(List<Operator<? extends OperatorDesc>> childOperators) |
static void |
OperatorUtils.setChildrenCollector(List<Operator<? extends OperatorDesc>> childOperators,
Map<String,org.apache.hadoop.mapred.OutputCollector> outMap) |
static void |
OperatorUtils.setChildrenCollector(List<Operator<? extends OperatorDesc>> childOperators,
org.apache.hadoop.mapred.OutputCollector out) |
static void |
OperatorUtils.setMemoryAvailable(List<Operator<? extends OperatorDesc>> operators,
long memoryAvailableToTask) |
void |
Operator.setParentOperators(List<Operator<? extends OperatorDesc>> parentOperators) |
Constructor and Description |
---|
FetchOperator(FetchWork work,
org.apache.hadoop.mapred.JobConf job,
Operator<?> operator,
List<VirtualColumn> vcCols) |
MapOpCtx(String alias,
Operator<?> op,
PartitionDesc partDesc) |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
MapRedTask.getReducer(MapWork mapWork) |
Modifier and Type | Method and Description |
---|---|
Collection<Operator<? extends OperatorDesc>> |
MapredLocalTask.getTopOperators() |
Collection<Operator<? extends OperatorDesc>> |
ExecDriver.getTopOperators() |
Modifier and Type | Method and Description |
---|---|
void |
ExecMapper.ReportStats.func(Operator op) |
Modifier and Type | Method and Description |
---|---|
static <T extends Operator<?>> |
SparkUtilities.collectOp(Operator<?> root,
Class<T> cls,
Collection<T> result,
Set<Operator<?>> seen)
Collect operators of type T starting from root.
|
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
SparkTask.getReducer(MapWork mapWork) |
Modifier and Type | Method and Description |
---|---|
static void |
SparkUtilities.collectOp(Collection<Operator<?>> result,
Operator<?> root,
Class<?> clazz)
Recursively find all operators under root, that are of class clazz or are the sub-class of clazz, and
put them in result.
|
static <T extends Operator<?>> |
SparkUtilities.collectOp(Operator<?> root,
Class<T> cls,
Collection<T> result,
Set<Operator<?>> seen)
Collect operators of type T starting from root.
|
static SparkPartitionPruningSinkOperator |
SparkUtilities.findReusableDPPSink(Operator<? extends OperatorDesc> branchingOP,
List<Operator<? extends OperatorDesc>> list) |
static boolean |
SparkUtilities.isDirectDPPBranch(Operator<?> op) |
Modifier and Type | Method and Description |
---|---|
static void |
SparkUtilities.collectOp(Collection<Operator<?>> result,
Operator<?> root,
Class<?> clazz)
Recursively find all operators under root, that are of class clazz or are the sub-class of clazz, and
put them in result.
|
static <T extends Operator<?>> |
SparkUtilities.collectOp(Operator<?> root,
Class<T> cls,
Collection<T> result,
Set<Operator<?>> seen)
Collect operators of type T starting from root.
|
static SparkPartitionPruningSinkOperator |
SparkUtilities.findReusableDPPSink(Operator<? extends OperatorDesc> branchingOP,
List<Operator<? extends OperatorDesc>> list) |
Modifier and Type | Field and Description |
---|---|
protected Operator<? extends OperatorDesc> |
MergeFileRecordProcessor.mergeOp |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
TezTask.getReducer(MapWork mapWork) |
Modifier and Type | Class and Description |
---|---|
class |
VectorAppMasterEventOperator
App Master Event operator implementation.
|
class |
VectorFileSinkOperator
File Sink operator implementation.
|
class |
VectorFilterOperator
Filter operator implementation.
|
class |
VectorGroupByOperator
Vectorized GROUP BY operator implementation.
|
class |
VectorLimitOperator
Limit operator implementation Limits the number of rows to be passed on.
|
class |
VectorMapJoinBaseOperator
The *NON-NATIVE* base vector map join operator class used by VectorMapJoinOperator and
VectorMapJoinOuterFilteredOperator.
|
class |
VectorMapJoinOperator
The vectorized version of the MapJoinOperator.
|
class |
VectorMapJoinOuterFilteredOperator
This is the *NON-NATIVE* vector map join operator for just LEFT OUTER JOIN and filtered.
|
class |
VectorMapOperator |
class |
VectorReduceSinkOperator |
class |
VectorSelectOperator
Select operator implementation.
|
class |
VectorSMBMapJoinOperator
VectorSMBJoinOperator.
|
class |
VectorSparkHashTableSinkOperator
Vectorized version of SparkHashTableSinkOperator
Currently the implementation just delegates all the work to super class
Copied from VectorFileSinkOperator
|
class |
VectorSparkPartitionPruningSinkOperator
Vectorized version for SparkPartitionPruningSinkOperator.
|
Modifier and Type | Method and Description |
---|---|
void |
VectorMapOperator.initEmptyInputChildren(List<Operator<?>> children,
org.apache.hadoop.conf.Configuration hconf) |
Modifier and Type | Class and Description |
---|---|
class |
VectorPTFOperator
This class is native vectorized PTF operator class.
|
Modifier and Type | Class and Description |
---|---|
class |
VectorReduceSinkCommonOperator
This class is common operator class for native vectorized reduce sink.
|
class |
VectorReduceSinkEmptyKeyOperator
This class is the UniformHash empty key operator class for native vectorized reduce sink.
|
class |
VectorReduceSinkLongOperator |
class |
VectorReduceSinkMultiKeyOperator |
class |
VectorReduceSinkObjectHashOperator
This class is the object hash (not Uniform Hash) operator class for native vectorized reduce sink.
|
class |
VectorReduceSinkStringOperator |
class |
VectorReduceSinkUniformHashOperator
This class is uniform hash (common) operator class for native vectorized reduce sink.
|
Modifier and Type | Method and Description |
---|---|
static List<Operator<? extends OperatorDesc>> |
HiveFileFormatUtils.doGetWorksFromPath(Map<org.apache.hadoop.fs.Path,ArrayList<String>> pathToAliases,
Map<String,Operator<? extends OperatorDesc>> aliasToWork,
org.apache.hadoop.fs.Path dir)
Get the list of operators from the operator tree that are needed for the path
|
Modifier and Type | Method and Description |
---|---|
static List<Operator<? extends OperatorDesc>> |
HiveFileFormatUtils.doGetWorksFromPath(Map<org.apache.hadoop.fs.Path,ArrayList<String>> pathToAliases,
Map<String,Operator<? extends OperatorDesc>> aliasToWork,
org.apache.hadoop.fs.Path dir)
Get the list of operators from the operator tree that are needed for the path
|
Modifier and Type | Method and Description |
---|---|
static Set<Operator<?>> |
GenMapRedUtils.findTopOps(Operator<?> startOp,
Class<?> clazz) |
LinkedHashMap<Operator<? extends OperatorDesc>,GenMRProcContext.GenMapRedCtx> |
GenMRProcContext.getMapCurrCtx() |
HashMap<Operator<? extends OperatorDesc>,Task<? extends Serializable>> |
GenMRProcContext.getOpTaskMap() |
Map<Operator<? extends Serializable>,Map<ColumnInfo,ExprNodeDesc>> |
ConstantPropagateProcCtx.getOpToConstantExprs() |
Set<Operator<? extends Serializable>> |
ConstantPropagateProcCtx.getOpToDelete() |
Map<Operator<? extends OperatorDesc>,List<FieldNode>> |
ColumnPrunerProcCtx.getPrunedColLists() |
Modifier and Type | Method and Description |
---|---|
void |
ConstantPropagateProcCtx.addOpToDelete(Operator<? extends Serializable> op) |
void |
GenMRProcContext.addSeenOp(Task task,
Operator operator) |
static TableScanOperator |
GenMapRedUtils.createTemporaryFile(Operator<? extends OperatorDesc> parent,
Operator<? extends OperatorDesc> child,
org.apache.hadoop.fs.Path taskTmpDir,
TableDesc tt_desc,
ParseContext parseCtx)
Break the pipeline between parent and child, and then
output data generated by parent to a temporary file stored in taskTmpDir.
|
static TableScanOperator |
GenMapRedUtils.createTemporaryFile(Operator<? extends OperatorDesc> parent,
Operator<? extends OperatorDesc> child,
org.apache.hadoop.fs.Path taskTmpDir,
TableDesc tt_desc,
ParseContext parseCtx)
Break the pipeline between parent and child, and then
output data generated by parent to a temporary file stored in taskTmpDir.
|
static String |
GenMapRedUtils.findAlias(MapWork work,
Operator<?> operator) |
static Set<String> |
GenMapRedUtils.findAliases(MapWork work,
Operator<?> startOp) |
static Set<Operator<?>> |
GenMapRedUtils.findTopOps(Operator<?> startOp,
Class<?> clazz) |
List<FieldNode> |
ColumnPrunerProcCtx.genColLists(Operator<? extends OperatorDesc> curOp)
Creates the list of internal column names(represented by field nodes,
these names are used in the RowResolver and are different from the
external column names) that are needed in the subtree.
|
List<FieldNode> |
ColumnPrunerProcCtx.genColLists(Operator<? extends OperatorDesc> curOp,
Operator<? extends OperatorDesc> child)
Creates the list of internal column names (represented by field nodes,
these names are used in the RowResolver and are different from the
external column names) that are needed in the subtree.
|
List<FieldNode> |
ColumnPrunerProcCtx.genColLists(Operator<? extends OperatorDesc> curOp,
Operator<? extends OperatorDesc> child)
Creates the list of internal column names (represented by field nodes,
these names are used in the RowResolver and are different from the
external column names) that are needed in the subtree.
|
protected void |
SizeBasedBigTableSelectorForAutoSMJ.getListTopOps(Operator<? extends OperatorDesc> op,
List<TableScanOperator> topOps) |
static BaseWork |
ReduceSinkMapJoinProc.getMapJoinParentWork(GenTezProcContext context,
Operator<?> parentRS) |
Map<ColumnInfo,ExprNodeDesc> |
ConstantPropagateProcCtx.getPropagatedConstants(Operator<? extends Serializable> op)
Get propagated constant map from parents.
|
List<FieldNode> |
ColumnPrunerProcCtx.getPrunedColList(Operator<? extends OperatorDesc> op) |
void |
ColumnPrunerProcCtx.handleFilterUnionChildren(Operator<? extends OperatorDesc> curOp)
If the input filter operator has direct child(ren) which are union operator,
and the filter's column is not the same as union's
create select operator between them.
|
boolean |
GenMRProcContext.isSeenOp(Task task,
Operator operator) |
boolean |
GroupByOptimizer.SortGroupByProcessor.removeChildren(Operator<? extends OperatorDesc> currOp,
int depth) |
static ListSinkOperator |
SimpleFetchOptimizer.replaceFSwithLS(Operator<?> fileSink,
String nullFormat) |
static void |
GenMapRedUtils.setKeyAndValueDesc(ReduceWork plan,
Operator<? extends OperatorDesc> topOp)
set key and value descriptor.
|
static void |
GenMapRedUtils.setTaskPlan(org.apache.hadoop.fs.Path path,
String alias,
Operator<? extends OperatorDesc> topOp,
MapWork plan,
boolean local,
TableDesc tt_desc)
set the current task in the mapredWork.
|
Modifier and Type | Method and Description |
---|---|
protected boolean |
AbstractBucketJoinProc.checkConvertBucketMapJoin(BucketJoinProcCtx context,
Map<String,Operator<? extends OperatorDesc>> aliasToOpInfo,
Map<Byte,List<ExprNodeDesc>> keysMap,
String baseBigAlias,
List<String> joinAliases) |
void |
GenMRProcContext.setMapCurrCtx(LinkedHashMap<Operator<? extends OperatorDesc>,GenMRProcContext.GenMapRedCtx> mapCurrCtx) |
void |
GenMRProcContext.setOpTaskMap(HashMap<Operator<? extends OperatorDesc>,Task<? extends Serializable>> opTaskMap) |
Constructor and Description |
---|
GenMRProcContext(HiveConf conf,
HashMap<Operator<? extends OperatorDesc>,Task<? extends Serializable>> opTaskMap,
ParseContext parseCtx,
List<Task<MoveWork>> mvTask,
List<Task<? extends Serializable>> rootTasks,
LinkedHashMap<Operator<? extends OperatorDesc>,GenMRProcContext.GenMapRedCtx> mapCurrCtx,
Set<ReadEntity> inputs,
Set<WriteEntity> outputs) |
GenMRProcContext(HiveConf conf,
HashMap<Operator<? extends OperatorDesc>,Task<? extends Serializable>> opTaskMap,
ParseContext parseCtx,
List<Task<MoveWork>> mvTask,
List<Task<? extends Serializable>> rootTasks,
LinkedHashMap<Operator<? extends OperatorDesc>,GenMRProcContext.GenMapRedCtx> mapCurrCtx,
Set<ReadEntity> inputs,
Set<WriteEntity> outputs) |
Modifier and Type | Method and Description |
---|---|
Operator |
HiveOpConverter.convert(org.apache.calcite.rel.RelNode root) |
Modifier and Type | Method and Description |
---|---|
protected static <T extends Operator<?>> |
CorrelationUtilities.findFirstPossibleParent(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findFirstPossibleParentPreserveSortOrder(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findParents(JoinOperator join,
Class<T> target) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findPossibleParent(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findPossibleParents(Operator<?> start,
Class<T> target,
boolean trustScript) |
Modifier and Type | Method and Description |
---|---|
protected static <T extends Operator<?>> |
CorrelationUtilities.findParents(JoinOperator join,
Class<T> target) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findPossibleParents(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static Operator<?> |
CorrelationUtilities.getSingleChild(Operator<?> operator) |
protected static Operator<?> |
CorrelationUtilities.getSingleChild(Operator<?> operator,
boolean throwException) |
protected static Operator<?> |
CorrelationUtilities.getSingleParent(Operator<?> operator) |
protected static Operator<?> |
CorrelationUtilities.getSingleParent(Operator<?> operator,
boolean throwException) |
protected static Operator<?> |
CorrelationUtilities.getStartForGroupBy(ReduceSinkOperator cRS,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx) |
Modifier and Type | Method and Description |
---|---|
static List<Operator<? extends OperatorDesc>> |
CorrelationUtilities.findSiblingOperators(Operator<? extends OperatorDesc> op)
Find all sibling operators (which have the same child operator of op) of op (op
included).
|
Modifier and Type | Method and Description |
---|---|
protected static <T extends Operator<?>> |
CorrelationUtilities.findFirstPossibleParent(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findFirstPossibleParentPreserveSortOrder(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findPossibleParent(Operator<?> start,
Class<T> target,
boolean trustScript) |
protected static <T extends Operator<?>> |
CorrelationUtilities.findPossibleParents(Operator<?> start,
Class<T> target,
boolean trustScript) |
static List<Operator<? extends OperatorDesc>> |
CorrelationUtilities.findSiblingOperators(Operator<? extends OperatorDesc> op)
Find all sibling operators (which have the same child operator of op) of op (op
included).
|
protected static Set<TableScanOperator> |
CorrelationUtilities.findTableScanOperators(Operator<?> startPoint)
Search the query plan tree from startPoint to the bottom.
|
protected static Operator<?> |
CorrelationUtilities.getSingleChild(Operator<?> operator) |
protected static Operator<?> |
CorrelationUtilities.getSingleChild(Operator<?> operator,
boolean throwException) |
protected static <T> T |
CorrelationUtilities.getSingleChild(Operator<?> operator,
Class<T> type) |
protected static Operator<?> |
CorrelationUtilities.getSingleParent(Operator<?> operator) |
protected static Operator<?> |
CorrelationUtilities.getSingleParent(Operator<?> operator,
boolean throwException) |
protected static <T> T |
CorrelationUtilities.getSingleParent(Operator<?> operator,
Class<T> type) |
protected static int |
CorrelationUtilities.indexOf(ExprNodeDesc cexpr,
ExprNodeDesc[] pexprs,
Operator child,
Operator[] parents,
boolean[] sorted) |
protected static int |
CorrelationUtilities.indexOf(ExprNodeDesc cexpr,
ExprNodeDesc[] pexprs,
Operator child,
Operator[] parents,
boolean[] sorted) |
protected static void |
CorrelationUtilities.insertOperatorBetween(Operator<?> newOperator,
Operator<?> parent,
Operator<?> child) |
protected static void |
CorrelationUtilities.insertOperatorBetween(Operator<?> newOperator,
Operator<?> parent,
Operator<?> child) |
protected static void |
CorrelationUtilities.insertOperatorBetween(Operator<?> newOperator,
Operator<?> parent,
Operator<?> child) |
protected static void |
CorrelationUtilities.isNullOperator(Operator<?> operator)
Throws an exception if the input operator is null
|
protected static void |
CorrelationUtilities.removeOperator(Operator<?> target,
Operator<?> child,
Operator<?> parent,
ParseContext context) |
protected static void |
CorrelationUtilities.removeOperator(Operator<?> target,
Operator<?> child,
Operator<?> parent,
ParseContext context) |
protected static void |
CorrelationUtilities.removeOperator(Operator<?> target,
Operator<?> child,
Operator<?> parent,
ParseContext context) |
static void |
CorrelationUtilities.removeOperator(Operator<?> target,
ParseContext context) |
protected static Integer |
ReduceSinkDeDuplicationUtils.sameKeys(List<ExprNodeDesc> cexprs,
List<ExprNodeDesc> pexprs,
Operator<?> child,
Operator<?> parent) |
protected static Integer |
ReduceSinkDeDuplicationUtils.sameKeys(List<ExprNodeDesc> cexprs,
List<ExprNodeDesc> pexprs,
Operator<?> child,
Operator<?> parent) |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
ExprProcCtx.getInputOperator()
Gets the input operator.
|
protected static Operator<? extends OperatorDesc> |
OpProcFactory.getParent(Stack<Node> stack)
Returns the parent operator in the walk path to the current operator.
|
Modifier and Type | Method and Description |
---|---|
void |
LineageCtx.Index.addFinalSelectOp(SelectOperator sop,
Operator<? extends OperatorDesc> sinkOp) |
void |
LineageCtx.Index.addPredicate(Operator<? extends OperatorDesc> op,
LineageInfo.Predicate cond) |
void |
LineageCtx.Index.copyPredicates(Operator<? extends OperatorDesc> srcOp,
Operator<? extends OperatorDesc> tgtOp) |
void |
LineageCtx.Index.copyPredicates(Operator<? extends OperatorDesc> srcOp,
Operator<? extends OperatorDesc> tgtOp) |
Map<ColumnInfo,LineageInfo.Dependency> |
LineageCtx.Index.getDependencies(Operator<? extends OperatorDesc> op) |
LineageInfo.Dependency |
LineageCtx.Index.getDependency(Operator<? extends OperatorDesc> op,
ColumnInfo col)
Gets the dependency for an operator, columninfo tuple.
|
LineageInfo.Dependency |
LineageCtx.Index.getDependency(Operator<? extends OperatorDesc> op,
String internalName)
Gets the dependency for a tuple of an operator,
and a ColumnInfo with specified internal name.
|
static LineageInfo.Dependency |
ExprProcFactory.getExprDependency(LineageCtx lctx,
Operator<? extends OperatorDesc> inpOp,
ExprNodeDesc expr)
Gets the expression dependencies for the expression.
|
static LineageInfo.Dependency |
ExprProcFactory.getExprDependency(LineageCtx lctx,
Operator<? extends OperatorDesc> inpOp,
ExprNodeDesc expr,
HashMap<Node,Object> outputMap)
Gets the expression dependencies for the expression.
|
static String |
ExprProcFactory.getExprString(RowSchema rs,
ExprNodeDesc expr,
LineageCtx lctx,
Operator<? extends OperatorDesc> inpOp,
LineageInfo.Predicate cond)
Get the expression string of an expression node.
|
Set<LineageInfo.Predicate> |
LineageCtx.Index.getPredicates(Operator<? extends OperatorDesc> op) |
void |
LineageCtx.Index.mergeDependency(Operator<? extends OperatorDesc> op,
ColumnInfo ci,
LineageInfo.Dependency dep)
Merges the new dependencies in dep to the existing dependencies
of (op, ci).
|
void |
LineageCtx.Index.putDependency(Operator<? extends OperatorDesc> op,
ColumnInfo col,
LineageInfo.Dependency dep)
Puts the dependency for an operator, columninfo tuple.
|
Constructor and Description |
---|
ExprProcCtx(LineageCtx lctx,
Operator<? extends OperatorDesc> inpOp)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
PcrOpWalkerCtx.OpToDeleteInfo.getParent() |
Constructor and Description |
---|
OpToDeleteInfo(Operator<? extends OperatorDesc> parent,
FilterOperator operator) |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
VectorizerReason.getOperator() |
protected static Operator<? extends OperatorDesc> |
BucketingSortingOpProcFactory.getParent(Stack<Node> stack)
Returns the parent operator in the walk path to the current operator.
|
Operator<? extends OperatorDesc> |
Vectorizer.validateAndVectorizeOperator(Operator<? extends OperatorDesc> op,
VectorizationContext vContext,
boolean isReduce,
boolean isTezOrSpark,
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.VectorTaskColumnInfo vectorTaskColumnInfo) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeFilterOperator(Operator<? extends OperatorDesc> filterOp,
VectorizationContext vContext,
VectorFilterDesc vectorFilterDesc) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeGroupByOperator(Operator<? extends OperatorDesc> groupByOp,
VectorizationContext vContext,
VectorGroupByDesc vectorGroupByDesc) |
Operator<? extends OperatorDesc> |
Vectorizer.vectorizeOperator(Operator<? extends OperatorDesc> op,
VectorizationContext vContext,
boolean isReduce,
boolean isTezOrSpark,
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.VectorTaskColumnInfo vectorTaskColumnInfo) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizePTFOperator(Operator<? extends OperatorDesc> ptfOp,
VectorizationContext vContext,
VectorPTFDesc vectorPTFDesc) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeSelectOperator(Operator<? extends OperatorDesc> selectOp,
VectorizationContext vContext,
VectorSelectDesc vectorSelectDesc) |
Modifier and Type | Method and Description |
---|---|
Map<MapJoinOperator,List<Operator<? extends OperatorDesc>>> |
MapJoinResolver.LocalMapJoinProcCtx.getDirectWorks() |
List<Operator<? extends OperatorDesc>> |
MapJoinResolver.LocalMapJoinProcCtx.getDummyParentOp() |
Modifier and Type | Method and Description |
---|---|
void |
MapJoinResolver.LocalMapJoinProcCtx.addDummyParentOp(Operator<? extends OperatorDesc> op) |
static VectorizerReason |
VectorizerReason.createExpressionIssue(Operator<? extends OperatorDesc> operator,
String expressionTitle,
String issue) |
static VectorizerReason |
VectorizerReason.createOperatorIssue(Operator<? extends OperatorDesc> operator,
String issue) |
List<BucketingSortingCtx.BucketCol> |
BucketingSortingCtx.getBucketedCols(Operator<? extends OperatorDesc> op) |
List<BucketingSortingCtx.SortCol> |
BucketingSortingCtx.getSortedCols(Operator<? extends OperatorDesc> op) |
void |
LocalMapJoinProcFactory.LocalMapJoinProcessor.hasGroupBy(Operator<? extends OperatorDesc> mapJoinOp,
MapJoinResolver.LocalMapJoinProcCtx localMapJoinProcCtx) |
protected Object |
BucketingSortingOpProcFactory.GroupByInferrer.processGroupBy(Operator<? extends OperatorDesc> parent,
GroupByOperator gop,
BucketingSortingCtx bctx)
Process a GroupByOperator to determine which if any columns the output is bucketed and
sorted by, assumes the columns output by the parent which are bucketed and sorted have
already been determined.
|
void |
BucketingSortingCtx.setBucketedCols(Operator<? extends OperatorDesc> op,
List<BucketingSortingCtx.BucketCol> bucketCols) |
void |
BucketingSortingCtx.setSortedCols(Operator<? extends OperatorDesc> op,
List<BucketingSortingCtx.SortCol> sortedCols) |
Operator<? extends OperatorDesc> |
Vectorizer.validateAndVectorizeOperator(Operator<? extends OperatorDesc> op,
VectorizationContext vContext,
boolean isReduce,
boolean isTezOrSpark,
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.VectorTaskColumnInfo vectorTaskColumnInfo) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeFilterOperator(Operator<? extends OperatorDesc> filterOp,
VectorizationContext vContext,
VectorFilterDesc vectorFilterDesc) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeGroupByOperator(Operator<? extends OperatorDesc> groupByOp,
VectorizationContext vContext,
VectorGroupByDesc vectorGroupByDesc) |
Operator<? extends OperatorDesc> |
Vectorizer.vectorizeOperator(Operator<? extends OperatorDesc> op,
VectorizationContext vContext,
boolean isReduce,
boolean isTezOrSpark,
org.apache.hadoop.hive.ql.optimizer.physical.Vectorizer.VectorTaskColumnInfo vectorTaskColumnInfo) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizePTFOperator(Operator<? extends OperatorDesc> ptfOp,
VectorizationContext vContext,
VectorPTFDesc vectorPTFDesc) |
static Operator<? extends OperatorDesc> |
Vectorizer.vectorizeSelectOperator(Operator<? extends OperatorDesc> selectOp,
VectorizationContext vContext,
VectorSelectDesc vectorSelectDesc) |
Modifier and Type | Method and Description |
---|---|
void |
MapJoinResolver.LocalMapJoinProcCtx.addDirectWorks(MapJoinOperator mapJoinOp,
List<Operator<? extends OperatorDesc>> directOperators) |
void |
AnnotateRunTimeStatsOptimizer.resolve(Set<Operator<?>> opSet,
ParseContext pctx) |
void |
MapJoinResolver.LocalMapJoinProcCtx.setDirectWorks(Map<MapJoinOperator,List<Operator<? extends OperatorDesc>>> directWorks) |
void |
MapJoinResolver.LocalMapJoinProcCtx.setDummyParentOp(List<Operator<? extends OperatorDesc>> op) |
static void |
AnnotateRunTimeStatsOptimizer.setOrAnnotateStats(Set<Operator<? extends OperatorDesc>> ops,
ParseContext pctx) |
Modifier and Type | Method and Description |
---|---|
OpTreeSignature |
OpTreeSignatureFactory.getSignature(Operator<? extends OperatorDesc> op) |
OpTreeSignature |
OpTreeSignatureFactory.Direct.getSignature(Operator<? extends OperatorDesc> op) |
OpTreeSignature |
OpTreeSignatureFactory.CachedFactory.getSignature(Operator<? extends OperatorDesc> op) |
static OpTreeSignature |
OpTreeSignature.of(Operator<?> root) |
static OpSignature |
OpSignature.of(Operator<? extends OperatorDesc> op) |
static OpTreeSignature |
OpTreeSignature.of(Operator<? extends OperatorDesc> op,
OpTreeSignatureFactory osf) |
Modifier and Type | Method and Description |
---|---|
Map<Operator<?>,ReduceWork> |
SparkSkewJoinResolver.SparkSkewJoinProcCtx.getReducerToReduceWork() |
Modifier and Type | Method and Description |
---|---|
protected long |
StatsRulesProcFactory.FilterStatsRule.evaluateExpression(Statistics stats,
ExprNodeDesc pred,
AnnotateStatsProcCtx aspCtx,
List<String> neededCols,
Operator<?> op,
long currNumRows) |
Modifier and Type | Field and Description |
---|---|
Operator<? extends OperatorDesc> |
GenTezProcContext.currentRootOperator |
Operator<? extends OperatorDesc> |
GenTezProcContext.parentOfRoot |
Modifier and Type | Method and Description |
---|---|
protected Operator |
SemanticAnalyzer.genFileSinkPlan(String dest,
QB qb,
Operator input) |
Operator |
SemanticAnalyzer.genPlan(QB qb) |
Operator |
SemanticAnalyzer.genPlan(QB qb,
boolean skipAmbiguityCheck) |
Operator<?> |
ParseContext.getFetchSource() |
<T extends OperatorDesc> |
SemanticAnalyzer.putOpInsertMap(Operator<T> op,
RowResolver rr) |
Modifier and Type | Method and Description |
---|---|
Map<String,Operator<? extends OperatorDesc>> |
QBJoinTree.getAliasToOpInfo() |
Collection<Operator> |
ParseContext.getAllOps() |
Map<Operator<? extends OperatorDesc>,Map<String,List<String>>> |
TableAccessInfo.getOperatorToTableAccessMap() |
Modifier and Type | Method and Description |
---|---|
void |
TableAccessInfo.add(Operator<? extends OperatorDesc> op,
Map<String,List<String>> tableToKeysMap) |
void |
TableAccessCtx.addOperatorTableAccess(Operator<? extends OperatorDesc> op,
Map<String,List<String>> tableToKeysMap) |
static boolean |
ColumnStatsAutoGatherContext.canRunAutogatherStats(Operator curr) |
MapWork |
GenTezUtils.createMapWork(GenTezProcContext context,
Operator<?> root,
TezWork tezWork,
PrunedPartitionList partitions) |
static ReduceWork |
GenTezUtils.createReduceWork(GenTezProcContext context,
Operator<?> root,
TezWork tezWork) |
static UnionWork |
GenTezUtils.createUnionWork(GenTezProcContext context,
Operator<?> root,
Operator<?> leaf,
TezWork tezWork) |
static UnionWork |
GenTezUtils.createUnionWork(GenTezProcContext context,
Operator<?> root,
Operator<?> leaf,
TezWork tezWork) |
protected Operator |
SemanticAnalyzer.genFileSinkPlan(String dest,
QB qb,
Operator input) |
static TableScanOperator |
TableAccessAnalyzer.genRootTableScan(Operator<? extends OperatorDesc> op,
List<String> keyNames)
This method traces up from the given operator to the root
of the operator graph until a TableScanOperator is reached.
|
static BaseWork |
GenTezUtils.getEnclosingWork(Operator<?> op,
GenTezProcContext procCtx)
getEncosingWork finds the BaseWork any given operator belongs to.
|
<T extends OperatorDesc> |
SemanticAnalyzer.putOpInsertMap(Operator<T> op,
RowResolver rr) |
static void |
GenTezUtils.removeBranch(Operator<?> event)
Remove an operator branch.
|
void |
ParseContext.setFetchSource(Operator<?> fetchSource) |
Modifier and Type | Method and Description |
---|---|
void |
QBJoinTree.setAliasToOpInfo(Map<String,Operator<? extends OperatorDesc>> aliasToOpInfo) |
void |
OptimizeTezProcContext.setRootOperators(Deque<Operator<? extends OperatorDesc>> roots) |
Constructor and Description |
---|
ColumnStatsAutoGatherContext(SemanticAnalyzer sa,
HiveConf conf,
Operator<? extends OperatorDesc> op,
Table tbl,
Map<String,String> partSpec,
boolean isInsertInto,
Context ctx) |
Modifier and Type | Class and Description |
---|---|
class |
SparkPartitionPruningSinkOperator
This operator gets partition info from the upstream operators, and write them
to HDFS.
|
Modifier and Type | Field and Description |
---|---|
Operator<? extends OperatorDesc> |
GenSparkProcContext.currentRootOperator |
Operator<? extends OperatorDesc> |
GenSparkProcContext.parentOfRoot |
Modifier and Type | Field and Description |
---|---|
Map<Operator<?>,List<BaseWork>> |
GenSparkProcContext.childToWorkMap |
Set<Operator<?>> |
GenSparkProcContext.clonedPruningTableScanSet |
Map<Operator<?>,List<Operator<?>>> |
GenSparkProcContext.linkChildOpWithDummyOp |
Map<Operator<?>,List<Operator<?>>> |
GenSparkProcContext.linkChildOpWithDummyOp |
Map<Operator<?>,Map<BaseWork,SparkEdgeProperty>> |
GenSparkProcContext.linkOpWithWorkMap |
Map<MapJoinOperator,List<Operator<?>>> |
GenSparkProcContext.mapJoinParentMap |
Set<Operator<?>> |
GenSparkProcContext.pruningSinkSet |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.rootToWorkMap |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.unionWorkMap |
Modifier and Type | Method and Description |
---|---|
Operator<?> |
SparkPartitionPruningSinkOperator.getBranchingOp() |
Modifier and Type | Method and Description |
---|---|
MapWork |
GenSparkUtils.createMapWork(GenSparkProcContext context,
Operator<?> root,
SparkWork sparkWork,
PrunedPartitionList partitions) |
MapWork |
GenSparkUtils.createMapWork(GenSparkProcContext context,
Operator<?> root,
SparkWork sparkWork,
PrunedPartitionList partitions,
boolean deferSetup) |
ReduceWork |
GenSparkUtils.createReduceWork(GenSparkProcContext context,
Operator<?> root,
SparkWork sparkWork) |
static <T> T |
GenSparkUtils.getChildOperator(Operator<?> root,
Class<T> klazz) |
Modifier and Type | Field and Description |
---|---|
Operator<?> |
ExprNodeDescUtils.ColumnOrigin.op |
Modifier and Type | Method and Description |
---|---|
Operator<?> |
MapredWork.getAnyOperator() |
Operator<?> |
MergeJoinWork.getAnyRootOperator() |
abstract Operator<? extends OperatorDesc> |
BaseWork.getAnyRootOperator() |
Operator<? extends OperatorDesc> |
UnionWork.getAnyRootOperator() |
Operator<? extends OperatorDesc> |
ReduceWork.getAnyRootOperator() |
Operator<? extends OperatorDesc> |
MapWork.getAnyRootOperator() |
Operator<?> |
ReduceWork.getReducer() |
static Operator<?> |
ExprNodeDescUtils.getSingleParent(Operator<?> current,
Operator<?> terminal) |
Operator<? extends OperatorDesc> |
ExprNodeDynamicListDesc.getSource() |
Operator<?> |
FetchWork.getSource() |
Modifier and Type | Method and Description |
---|---|
void |
MapWork.addMapWork(org.apache.hadoop.fs.Path path,
String alias,
Operator<?> work,
PartitionDesc pd) |
static ExprNodeDesc |
ExprNodeDescUtils.backtrack(ExprNodeDesc source,
Operator<?> current,
Operator<?> terminal) |
static ExprNodeDesc |
ExprNodeDescUtils.backtrack(ExprNodeDesc source,
Operator<?> current,
Operator<?> terminal) |
static ExprNodeDesc |
ExprNodeDescUtils.backtrack(ExprNodeDesc source,
Operator<?> current,
Operator<?> terminal,
boolean foldExpr) |
static ExprNodeDesc |
ExprNodeDescUtils.backtrack(ExprNodeDesc source,
Operator<?> current,
Operator<?> terminal,
boolean foldExpr) |
static ArrayList<ExprNodeDesc> |
ExprNodeDescUtils.backtrack(List<ExprNodeDesc> sources,
Operator<?> current,
Operator<?> terminal)
Convert expressions in current operator to those in terminal operator, which
is an ancestor of current or null (back to top operator).
|
static ArrayList<ExprNodeDesc> |
ExprNodeDescUtils.backtrack(List<ExprNodeDesc> sources,
Operator<?> current,
Operator<?> terminal)
Convert expressions in current operator to those in terminal operator, which
is an ancestor of current or null (back to top operator).
|
static ArrayList<ExprNodeDesc> |
ExprNodeDescUtils.backtrack(List<ExprNodeDesc> sources,
Operator<?> current,
Operator<?> terminal,
boolean foldExpr) |
static ArrayList<ExprNodeDesc> |
ExprNodeDescUtils.backtrack(List<ExprNodeDesc> sources,
Operator<?> current,
Operator<?> terminal,
boolean foldExpr) |
static boolean |
ExprNodeDescUtils.checkPrefixKeys(List<ExprNodeDesc> childKeys,
List<ExprNodeDesc> parentKeys,
Operator<? extends OperatorDesc> childOp,
Operator<? extends OperatorDesc> parentOp)
Checks whether the keys of a parent operator are a prefix of the keys of a
child operator.
|
static boolean |
ExprNodeDescUtils.checkPrefixKeys(List<ExprNodeDesc> childKeys,
List<ExprNodeDesc> parentKeys,
Operator<? extends OperatorDesc> childOp,
Operator<? extends OperatorDesc> parentOp)
Checks whether the keys of a parent operator are a prefix of the keys of a
child operator.
|
static boolean |
ExprNodeDescUtils.checkPrefixKeysUpstream(List<ExprNodeDesc> childKeys,
List<ExprNodeDesc> parentKeys,
Operator<? extends OperatorDesc> childOp,
Operator<? extends OperatorDesc> parentOp)
Checks whether the keys of a child operator are a prefix of the keys of a
parent operator.
|
static boolean |
ExprNodeDescUtils.checkPrefixKeysUpstream(List<ExprNodeDesc> childKeys,
List<ExprNodeDesc> parentKeys,
Operator<? extends OperatorDesc> childOp,
Operator<? extends OperatorDesc> parentOp)
Checks whether the keys of a child operator are a prefix of the keys of a
parent operator.
|
static ExprNodeDescUtils.ColumnOrigin |
ExprNodeDescUtils.findColumnOrigin(ExprNodeDesc expr,
Operator<?> op) |
static ExprNodeDesc |
ExprNodeDescUtils.findConstantExprOrigin(String dpCol,
Operator<? extends OperatorDesc> op) |
static ArrayList<ExprNodeDesc> |
ExprNodeDescUtils.genExprNodeDesc(Operator inputOp,
int startPos,
int endPos,
boolean addEmptyTabAlias,
boolean setColToNonVirtual)
Build ExprNodeColumnDesc for the projections in the input operator from
sartpos to endpos(both included).
|
static Operator<?> |
ExprNodeDescUtils.getSingleParent(Operator<?> current,
Operator<?> terminal) |
static Operator<?> |
ExprNodeDescUtils.getSingleParent(Operator<?> current,
Operator<?> terminal) |
static ExprNodeDesc |
ExprNodeDescUtils.resolveJoinKeysAsRSColumns(ExprNodeDesc source,
Operator<?> reduceSinkOp)
Join keys are expressions based on the select operator.
|
static List<ExprNodeDesc> |
ExprNodeDescUtils.resolveJoinKeysAsRSColumns(List<ExprNodeDesc> sourceList,
Operator<?> reduceSinkOp) |
void |
ReduceWork.setReducer(Operator<?> reducer) |
void |
FetchWork.setSource(Operator<?> source) |
void |
ExprNodeDynamicListDesc.setSource(Operator<? extends OperatorDesc> source) |
Modifier and Type | Method and Description |
---|---|
void |
MergeJoinWork.addMergedWork(BaseWork work,
BaseWork connectWork,
Map<Operator<?>,BaseWork> leafOperatorToFollowingWork) |
MapredLocalWork |
MapredLocalWork.extractDirectWorks(Map<MapJoinOperator,List<Operator<? extends OperatorDesc>>> directWorks) |
void |
MergeJoinWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
MergeJoinWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
abstract void |
BaseWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
abstract void |
BaseWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
UnionWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
UnionWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
ReduceWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
ReduceWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
MapWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
MapWork.replaceRoots(Map<Operator<?>,Operator<?>> replacementMap) |
void |
JoinDesc.setAliasToOpInfo(Map<String,Operator<? extends OperatorDesc>> aliasToOpInfo) |
void |
MapredLocalWork.setAliasToWork(LinkedHashMap<String,Operator<? extends OperatorDesc>> aliasToWork) |
void |
MapWork.setAliasToWork(LinkedHashMap<String,Operator<? extends OperatorDesc>> aliasToWork) |
void |
MapredLocalWork.setDirectFetchOp(Map<MapJoinOperator,List<Operator<? extends OperatorDesc>>> op) |
void |
MapredLocalWork.setDummyParentOp(List<Operator<? extends OperatorDesc>> op) |
Constructor and Description |
---|
ColumnOrigin(ExprNodeColumnDesc col,
Operator<?> op) |
ExprNodeDynamicListDesc(TypeInfo typeInfo,
Operator<? extends OperatorDesc> source,
int keyIndex) |
Constructor and Description |
---|
MapredLocalWork(LinkedHashMap<String,Operator<? extends OperatorDesc>> aliasToWork,
LinkedHashMap<String,FetchWork> aliasToFetchWork) |
MuxDesc(List<Operator<? extends OperatorDesc>> ops) |
Modifier and Type | Method and Description |
---|---|
OpTreeSignature |
PlanMapper.getSignatureOf(Operator<?> op) |
Modifier and Type | Method and Description |
---|---|
Operator<? extends OperatorDesc> |
ExprWalkerInfo.getOp() |
Modifier and Type | Method and Description |
---|---|
protected static Object |
OpProcFactory.createFilter(Operator op,
ExprWalkerInfo pushDownPreds,
OpWalkerInfo owi) |
protected static Object |
OpProcFactory.createFilter(Operator op,
Map<String,List<ExprNodeDesc>> predicates,
OpWalkerInfo owi) |
static ExprWalkerInfo |
ExprWalkerProcFactory.extractPushdownPreds(OpWalkerInfo opContext,
Operator<? extends OperatorDesc> op,
ExprNodeDesc pred) |
static ExprWalkerInfo |
ExprWalkerProcFactory.extractPushdownPreds(OpWalkerInfo opContext,
Operator<? extends OperatorDesc> op,
List<ExprNodeDesc> preds)
Extracts pushdown predicates from the given list of predicate expression.
|
ExprWalkerInfo |
OpWalkerInfo.getPrunedPreds(Operator<? extends OperatorDesc> op) |
ExprWalkerInfo |
OpWalkerInfo.putPrunedPreds(Operator<? extends OperatorDesc> op,
ExprWalkerInfo value) |
Constructor and Description |
---|
ExprWalkerInfo(Operator<? extends OperatorDesc> op) |
Modifier and Type | Method and Description |
---|---|
void |
LineageState.mapDirToOp(org.apache.hadoop.fs.Path dir,
Operator fop)
Adds a mapping from the load work to the file sink operator.
|
Modifier and Type | Method and Description |
---|---|
Operator<?> |
GenericUDAFBloomFilter.GenericUDAFBloomFilterEvaluator.getSourceOperator() |
Copyright © 2022 The Apache Software Foundation. All rights reserved.