Package | Description |
---|---|
org.apache.hadoop.hive.ql.exec |
Hive QL execution tasks, operators, functions and other handlers.
|
org.apache.hadoop.hive.ql.exec.spark | |
org.apache.hadoop.hive.ql.exec.tez | |
org.apache.hadoop.hive.ql.exec.tez.monitoring | |
org.apache.hadoop.hive.ql.io.merge | |
org.apache.hadoop.hive.ql.io.rcfile.truncate | |
org.apache.hadoop.hive.ql.optimizer | |
org.apache.hadoop.hive.ql.optimizer.physical | |
org.apache.hadoop.hive.ql.parse | |
org.apache.hadoop.hive.ql.parse.spark | |
org.apache.hadoop.hive.ql.plan |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
SerializationUtilities.cloneBaseWork(BaseWork plan)
Clones using the powers of XML.
|
static BaseWork |
Utilities.getMergeWork(org.apache.hadoop.conf.Configuration jconf) |
static BaseWork |
Utilities.getMergeWork(org.apache.hadoop.conf.Configuration jconf,
String prefix) |
Modifier and Type | Method and Description |
---|---|
Map<org.apache.hadoop.fs.Path,BaseWork> |
GlobalWorkMapFactory.get(org.apache.hadoop.conf.Configuration conf) |
Modifier and Type | Method and Description |
---|---|
static void |
Utilities.cacheBaseWork(org.apache.hadoop.conf.Configuration conf,
String name,
BaseWork work,
org.apache.hadoop.fs.Path hiveScratchDir) |
static BaseWork |
SerializationUtilities.cloneBaseWork(BaseWork plan)
Clones using the powers of XML.
|
static Set<Operator<?>> |
OperatorUtils.getOp(BaseWork work,
Class<?> clazz) |
static List<String> |
Utilities.getStatsTmpDirs(BaseWork work,
org.apache.hadoop.conf.Configuration conf) |
static void |
Utilities.setBaseWork(org.apache.hadoop.conf.Configuration conf,
String name,
BaseWork work)
Pushes work into the global work map
|
Modifier and Type | Method and Description |
---|---|
static String |
SparkUtilities.getWorkId(BaseWork work)
Return the ID for this BaseWork, in String form.
|
Modifier and Type | Field and Description |
---|---|
BaseWork |
DynamicValueRegistryTez.RegistryConfTez.baseWork |
Modifier and Type | Method and Description |
---|---|
List<BaseWork> |
RecordProcessor.getMergeWorkList(org.apache.hadoop.mapred.JobConf jconf,
String key,
String queryId,
ObjectCache cache,
List<String> cacheKeys) |
Modifier and Type | Method and Description |
---|---|
void |
DagUtils.addCredentials(BaseWork work,
org.apache.tez.dag.api.DAG dag)
Set up credentials for the base work on secure clusters
|
org.apache.tez.dag.api.Edge |
DagUtils.createEdge(org.apache.hadoop.mapred.JobConf vConf,
org.apache.tez.dag.api.Vertex v,
org.apache.tez.dag.api.Vertex w,
TezEdgeProperty edgeProp,
BaseWork work,
TezWork tezWork)
Given two vertices and the configuration for the source vertex, createEdge
will create an Edge object that connects the two.
|
org.apache.tez.dag.api.GroupInputEdge |
DagUtils.createEdge(org.apache.tez.dag.api.VertexGroup group,
org.apache.hadoop.mapred.JobConf vConf,
org.apache.tez.dag.api.Vertex w,
TezEdgeProperty edgeProp,
BaseWork work,
TezWork tezWork)
Given a Vertex group and a vertex createEdge will create an
Edge between them.
|
org.apache.tez.dag.api.Vertex |
DagUtils.createVertex(org.apache.hadoop.mapred.JobConf conf,
BaseWork work,
org.apache.hadoop.fs.Path scratchDir,
org.apache.hadoop.fs.FileSystem fileSystem,
Context ctx,
boolean hasChildren,
TezWork tezWork,
TezWork.VertexType vertexType,
Map<String,org.apache.hadoop.yarn.api.records.LocalResource> localResources)
Create a vertex from a given work object.
|
org.apache.hadoop.mapred.JobConf |
DagUtils.initializeVertexConf(org.apache.hadoop.mapred.JobConf conf,
Context context,
BaseWork work)
Creates and initializes the JobConf object for a given BaseWork object.
|
Constructor and Description |
---|
RegistryConfTez(org.apache.hadoop.conf.Configuration conf,
BaseWork baseWork,
org.apache.tez.runtime.api.ProcessorContext processorContext,
Map<String,org.apache.tez.runtime.api.LogicalInput> inputs) |
Constructor and Description |
---|
TezJobMonitor(List<BaseWork> topSortedWorks,
org.apache.tez.dag.api.client.DAGClient dagClient,
HiveConf conf,
org.apache.tez.dag.api.DAG dag,
Context ctx) |
Modifier and Type | Class and Description |
---|---|
class |
MergeFileWork |
Modifier and Type | Class and Description |
---|---|
class |
ColumnTruncateWork |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
ReduceSinkMapJoinProc.getMapJoinParentWork(GenTezProcContext context,
Operator<?> parentRS) |
Modifier and Type | Method and Description |
---|---|
void |
Vectorizer.testSetCurrentBaseWork(BaseWork testBaseWork) |
Modifier and Type | Field and Description |
---|---|
BaseWork |
GenTezProcContext.preceedingWork |
Modifier and Type | Field and Description |
---|---|
Map<Operator<?>,List<BaseWork>> |
GenTezProcContext.childToWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.leafOperatorToFollowingWork |
Map<Operator<?>,Map<BaseWork,TezEdgeProperty>> |
GenTezProcContext.linkOpWithWorkMap |
Map<BaseWork,List<ReduceSinkOperator>> |
GenTezProcContext.linkWorkWithReduceSinkMap |
Map<MapJoinOperator,List<BaseWork>> |
GenTezProcContext.mapJoinWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.rootToWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.unionWorkMap |
Set<BaseWork> |
GenTezProcContext.workWithUnionOperators |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
GenTezUtils.getEnclosingWork(Operator<?> op,
GenTezProcContext procCtx)
getEncosingWork finds the BaseWork any given operator belongs to.
|
Modifier and Type | Method and Description |
---|---|
static TezEdgeProperty.EdgeType |
GenTezUtils.determineEdgeType(BaseWork preceedingWork,
BaseWork followingWork,
ReduceSinkOperator reduceSinkOperator) |
static void |
GenTezUtils.removeUnionOperators(GenTezProcContext context,
BaseWork work,
int indexForTezUnion) |
Modifier and Type | Field and Description |
---|---|
BaseWork |
GenSparkProcContext.preceedingWork |
Modifier and Type | Field and Description |
---|---|
Map<Operator<?>,List<BaseWork>> |
GenSparkProcContext.childToWorkMap |
Map<Operator<?>,Map<BaseWork,SparkEdgeProperty>> |
GenSparkProcContext.linkOpWithWorkMap |
Map<BaseWork,List<ReduceSinkOperator>> |
GenSparkProcContext.linkWorkWithReduceSinkMap |
Map<MapJoinOperator,List<BaseWork>> |
GenSparkProcContext.mapJoinWorkMap |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.rootToWorkMap |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.unionWorkMap |
Set<BaseWork> |
GenSparkProcContext.workWithUnionOperators |
Modifier and Type | Method and Description |
---|---|
void |
GenSparkUtils.removeUnionOperators(GenSparkProcContext context,
BaseWork work) |
Modifier and Type | Class and Description |
---|---|
class |
MapWork
MapWork represents all the information used to run a map task on the cluster.
|
class |
MergeJoinWork |
class |
ReduceWork
ReduceWork represents all the information used to run a reduce task on the cluster.
|
class |
UnionWork
Simple wrapper for union all cases.
|
Modifier and Type | Field and Description |
---|---|
BaseWork |
SparkWork.Dependency.w |
BaseWork |
TezWork.Dependency.w |
Modifier and Type | Field and Description |
---|---|
protected Map<org.apache.commons.lang3.tuple.Pair<BaseWork,BaseWork>,SparkEdgeProperty> |
SparkWork.edgeProperties |
protected Map<org.apache.commons.lang3.tuple.Pair<BaseWork,BaseWork>,SparkEdgeProperty> |
SparkWork.edgeProperties |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.invertedWorkGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.invertedWorkGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.workGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.workGraph |
Modifier and Type | Method and Description |
---|---|
BaseWork |
MergeJoinWork.getMainWork() |
Modifier and Type | Method and Description |
---|---|
List<BaseWork> |
SparkWork.getAllWork() |
List<BaseWork> |
TezWork.getAllWork()
getAllWork returns a topologically sorted list of BaseWork
|
Collection<BaseWork> |
SparkWork.getAllWorkUnsorted() |
Collection<BaseWork> |
TezWork.getAllWorkUnsorted() |
List<BaseWork> |
MergeJoinWork.getBaseWorkList() |
List<BaseWork> |
SparkWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
List<BaseWork> |
TezWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
Map<BaseWork,BaseWork> |
SparkWork.getCloneToWork() |
Map<BaseWork,BaseWork> |
SparkWork.getCloneToWork() |
Set<BaseWork> |
SparkWork.getLeaves()
getLeaves returns all nodes that do not have a child
|
Set<BaseWork> |
TezWork.getLeaves()
getLeaves returns all nodes that do not have a child
|
List<BaseWork> |
SparkWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
List<BaseWork> |
TezWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
Set<BaseWork> |
SparkWork.getRoots()
getRoots returns all nodes that do not have a parent.
|
Set<BaseWork> |
TezWork.getRoots()
getRoots returns all nodes that do not have a parent.
|
Map<String,BaseWork> |
SparkWork.getWorkMap() |
Map<String,BaseWork> |
TezWork.getWorkMap()
getWorkMap returns a map of "vertex name" to BaseWork
|
Modifier and Type | Method and Description |
---|---|
void |
SparkWork.add(BaseWork w)
add creates a new node in the graph without any connections
|
void |
TezWork.add(BaseWork w)
add creates a new node in the graph without any connections
|
void |
SparkWork.addAll(BaseWork[] bws)
Add all nodes in the collection without any connections.
|
void |
TezWork.addAll(BaseWork[] bws)
add all nodes in the collection without any connections
|
void |
MergeJoinWork.addMergedWork(BaseWork work,
BaseWork connectWork,
Map<Operator<?>,BaseWork> leafOperatorToFollowingWork) |
void |
SparkWork.connect(BaseWork a,
BaseWork b,
SparkEdgeProperty edgeProp)
connect adds an edge between a and b.
|
void |
TezWork.connect(BaseWork a,
BaseWork b,
TezEdgeProperty edgeProp)
connect adds an edge between a and b.
|
boolean |
SparkWork.contains(BaseWork w)
Whether the specified BaseWork is a vertex in this graph
|
void |
SparkWork.disconnect(BaseWork a,
BaseWork b)
disconnect removes an edge between a and b.
|
void |
TezWork.disconnect(BaseWork a,
BaseWork b)
disconnect removes an edge between a and b.
|
List<BaseWork> |
SparkWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
List<BaseWork> |
TezWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
SparkEdgeProperty |
SparkWork.getEdgeProperty(BaseWork a,
BaseWork b)
returns the edge type connecting work a and b
|
TezEdgeProperty |
TezWork.getEdgeProperty(BaseWork a,
BaseWork b)
returns the edge type connecting work a and b
|
TezEdgeProperty.EdgeType |
TezWork.getEdgeType(BaseWork a,
BaseWork b) |
List<BaseWork> |
SparkWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
List<BaseWork> |
TezWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
TezWork.VertexType |
TezWork.getVertexType(BaseWork w) |
void |
SparkWork.remove(BaseWork work)
remove removes a node from the graph and removes all edges with
work as start or end point.
|
void |
TezWork.remove(BaseWork work)
remove removes a node from the graph and removes all edges with
work as start or end point.
|
void |
TezWork.setVertexType(BaseWork w,
TezWork.VertexType incomingVertexType) |
Modifier and Type | Method and Description |
---|---|
void |
SparkWork.addAll(Collection<BaseWork> c)
Add all nodes in the collection without any connections.
|
void |
TezWork.addAll(Collection<BaseWork> c)
add all nodes in the collection without any connections
|
void |
MergeJoinWork.addMergedWork(BaseWork work,
BaseWork connectWork,
Map<Operator<?>,BaseWork> leafOperatorToFollowingWork) |
void |
SparkWork.setCloneToWork(Map<BaseWork,BaseWork> cloneToWork) |
void |
SparkWork.setCloneToWork(Map<BaseWork,BaseWork> cloneToWork) |
Constructor and Description |
---|
BaseExplainVectorization(BaseWork baseWork) |
Copyright © 2022 The Apache Software Foundation. All rights reserved.