Package | Description |
---|---|
org.apache.hadoop.hive.ql.exec |
Hive QL execution tasks, operators, functions and other handlers.
|
org.apache.hadoop.hive.ql.exec.spark | |
org.apache.hadoop.hive.ql.exec.tez | |
org.apache.hadoop.hive.ql.exec.tez.monitoring | |
org.apache.hadoop.hive.ql.io.merge | |
org.apache.hadoop.hive.ql.io.rcfile.stats | |
org.apache.hadoop.hive.ql.io.rcfile.truncate | |
org.apache.hadoop.hive.ql.optimizer | |
org.apache.hadoop.hive.ql.optimizer.physical | |
org.apache.hadoop.hive.ql.parse | |
org.apache.hadoop.hive.ql.parse.spark | |
org.apache.hadoop.hive.ql.plan |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
SerializationUtilities.cloneBaseWork(BaseWork plan)
Clones using the powers of XML.
|
static BaseWork |
Utilities.getMergeWork(org.apache.hadoop.conf.Configuration jconf) |
static BaseWork |
Utilities.getMergeWork(org.apache.hadoop.conf.Configuration jconf,
String prefix) |
Modifier and Type | Method and Description |
---|---|
Map<org.apache.hadoop.fs.Path,BaseWork> |
GlobalWorkMapFactory.get(org.apache.hadoop.conf.Configuration conf) |
Modifier and Type | Method and Description |
---|---|
static void |
Utilities.cacheBaseWork(org.apache.hadoop.conf.Configuration conf,
String name,
BaseWork work,
org.apache.hadoop.fs.Path hiveScratchDir) |
static BaseWork |
SerializationUtilities.cloneBaseWork(BaseWork plan)
Clones using the powers of XML.
|
static List<String> |
Utilities.getStatsTmpDirs(BaseWork work,
org.apache.hadoop.conf.Configuration conf) |
static void |
Utilities.setBaseWork(org.apache.hadoop.conf.Configuration conf,
String name,
BaseWork work)
Pushes work into the global work map
|
Modifier and Type | Method and Description |
---|---|
static String |
SparkUtilities.getWorkId(BaseWork work)
Return the ID for this BaseWork, in String form.
|
Modifier and Type | Field and Description |
---|---|
BaseWork |
DynamicValueRegistryTez.RegistryConfTez.baseWork |
Modifier and Type | Method and Description |
---|---|
List<BaseWork> |
RecordProcessor.getMergeWorkList(org.apache.hadoop.mapred.JobConf jconf,
String key,
String queryId,
ObjectCache cache,
List<String> cacheKeys) |
Modifier and Type | Method and Description |
---|---|
void |
DagUtils.addCredentials(BaseWork work,
org.apache.tez.dag.api.DAG dag)
Set up credentials for the base work on secure clusters
|
org.apache.tez.dag.api.Vertex |
DagUtils.createVertex(org.apache.hadoop.mapred.JobConf conf,
BaseWork work,
org.apache.hadoop.fs.Path scratchDir,
org.apache.hadoop.yarn.api.records.LocalResource appJarLr,
List<org.apache.hadoop.yarn.api.records.LocalResource> additionalLr,
org.apache.hadoop.fs.FileSystem fileSystem,
Context ctx,
boolean hasChildren,
TezWork tezWork,
TezWork.VertexType vertexType)
Create a vertex from a given work object.
|
org.apache.hadoop.mapred.JobConf |
DagUtils.initializeVertexConf(org.apache.hadoop.mapred.JobConf conf,
Context context,
BaseWork work)
Creates and initializes the JobConf object for a given BaseWork object.
|
Constructor and Description |
---|
RegistryConfTez(org.apache.hadoop.conf.Configuration conf,
BaseWork baseWork,
org.apache.tez.runtime.api.ProcessorContext processorContext,
Map<String,org.apache.tez.runtime.api.LogicalInput> inputs) |
Constructor and Description |
---|
TezJobMonitor(Map<String,BaseWork> workMap,
org.apache.tez.dag.api.client.DAGClient dagClient,
HiveConf conf,
org.apache.tez.dag.api.DAG dag,
Context ctx) |
Modifier and Type | Class and Description |
---|---|
class |
MergeFileWork |
Modifier and Type | Class and Description |
---|---|
class |
PartialScanWork
Partial Scan Work.
|
Modifier and Type | Class and Description |
---|---|
class |
ColumnTruncateWork |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
ReduceSinkMapJoinProc.getMapJoinParentWork(GenTezProcContext context,
Operator<?> parentRS) |
Modifier and Type | Method and Description |
---|---|
void |
Vectorizer.debugDisplayAllMaps(BaseWork work) |
static Set<Operator<?>> |
SparkMapJoinResolver.getOp(BaseWork work,
Class<?> clazz) |
void |
Vectorizer.testSetCurrentBaseWork(BaseWork testBaseWork) |
Modifier and Type | Field and Description |
---|---|
BaseWork |
GenTezProcContext.preceedingWork |
Modifier and Type | Field and Description |
---|---|
Map<Operator<?>,List<BaseWork>> |
GenTezProcContext.childToWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.leafOperatorToFollowingWork |
Map<Operator<?>,Map<BaseWork,TezEdgeProperty>> |
GenTezProcContext.linkOpWithWorkMap |
Map<BaseWork,List<ReduceSinkOperator>> |
GenTezProcContext.linkWorkWithReduceSinkMap |
Map<MapJoinOperator,List<BaseWork>> |
GenTezProcContext.mapJoinWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.rootToWorkMap |
Map<Operator<?>,BaseWork> |
GenTezProcContext.unionWorkMap |
Set<BaseWork> |
GenTezProcContext.workWithUnionOperators |
Modifier and Type | Method and Description |
---|---|
static BaseWork |
GenTezUtils.getEnclosingWork(Operator<?> op,
GenTezProcContext procCtx)
getEncosingWork finds the BaseWork any given operator belongs to.
|
Modifier and Type | Method and Description |
---|---|
static TezEdgeProperty.EdgeType |
GenTezUtils.determineEdgeType(BaseWork preceedingWork,
BaseWork followingWork,
ReduceSinkOperator reduceSinkOperator) |
static void |
GenTezUtils.removeUnionOperators(GenTezProcContext context,
BaseWork work,
int indexForTezUnion) |
Modifier and Type | Field and Description |
---|---|
BaseWork |
GenSparkProcContext.preceedingWork |
Modifier and Type | Field and Description |
---|---|
Map<Operator<?>,List<BaseWork>> |
GenSparkProcContext.childToWorkMap |
Map<Operator<?>,Map<BaseWork,SparkEdgeProperty>> |
GenSparkProcContext.linkOpWithWorkMap |
Map<BaseWork,List<ReduceSinkOperator>> |
GenSparkProcContext.linkWorkWithReduceSinkMap |
Map<MapJoinOperator,List<BaseWork>> |
GenSparkProcContext.mapJoinWorkMap |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.rootToWorkMap |
Map<Operator<?>,BaseWork> |
GenSparkProcContext.unionWorkMap |
Set<BaseWork> |
GenSparkProcContext.workWithUnionOperators |
Modifier and Type | Method and Description |
---|---|
BaseWork |
GenSparkUtils.getEnclosingWork(Operator<?> op,
GenSparkProcContext procCtx)
getEncosingWork finds the BaseWork any given operator belongs to.
|
Modifier and Type | Method and Description |
---|---|
void |
GenSparkUtils.removeUnionOperators(GenSparkProcContext context,
BaseWork work) |
Modifier and Type | Class and Description |
---|---|
class |
MapWork
MapWork represents all the information used to run a map task on the cluster.
|
class |
MergeJoinWork |
class |
ReduceWork
ReduceWork represents all the information used to run a reduce task on the cluster.
|
class |
UnionWork
Simple wrapper for union all cases.
|
Modifier and Type | Field and Description |
---|---|
BaseWork |
SparkWork.Dependency.w |
BaseWork |
TezWork.Dependency.w |
Modifier and Type | Field and Description |
---|---|
protected Map<org.apache.commons.lang3.tuple.Pair<BaseWork,BaseWork>,SparkEdgeProperty> |
SparkWork.edgeProperties |
protected Map<org.apache.commons.lang3.tuple.Pair<BaseWork,BaseWork>,SparkEdgeProperty> |
SparkWork.edgeProperties |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.invertedWorkGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.invertedWorkGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.workGraph |
protected Map<BaseWork,List<BaseWork>> |
SparkWork.workGraph |
Modifier and Type | Method and Description |
---|---|
BaseWork |
MergeJoinWork.getMainWork() |
Modifier and Type | Method and Description |
---|---|
List<BaseWork> |
SparkWork.getAllWork() |
List<BaseWork> |
TezWork.getAllWork()
getAllWork returns a topologically sorted list of BaseWork
|
Collection<BaseWork> |
SparkWork.getAllWorkUnsorted() |
Collection<BaseWork> |
TezWork.getAllWorkUnsorted() |
List<BaseWork> |
MergeJoinWork.getBaseWorkList() |
List<BaseWork> |
SparkWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
List<BaseWork> |
TezWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
Map<BaseWork,BaseWork> |
SparkWork.getCloneToWork() |
Map<BaseWork,BaseWork> |
SparkWork.getCloneToWork() |
Set<BaseWork> |
SparkWork.getLeaves()
getLeaves returns all nodes that do not have a child
|
Set<BaseWork> |
TezWork.getLeaves()
getLeaves returns all nodes that do not have a child
|
List<BaseWork> |
SparkWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
List<BaseWork> |
TezWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
Set<BaseWork> |
SparkWork.getRoots()
getRoots returns all nodes that do not have a parent.
|
Set<BaseWork> |
TezWork.getRoots()
getRoots returns all nodes that do not have a parent.
|
Map<String,BaseWork> |
SparkWork.getWorkMap() |
Map<String,BaseWork> |
TezWork.getWorkMap()
getWorkMap returns a map of "vertex name" to BaseWork
|
Modifier and Type | Method and Description |
---|---|
void |
SparkWork.add(BaseWork w)
add creates a new node in the graph without any connections
|
void |
TezWork.add(BaseWork w)
add creates a new node in the graph without any connections
|
void |
SparkWork.addAll(BaseWork[] bws)
Add all nodes in the collection without any connections.
|
void |
TezWork.addAll(BaseWork[] bws)
add all nodes in the collection without any connections
|
void |
MergeJoinWork.addMergedWork(BaseWork work,
BaseWork connectWork,
Map<Operator<?>,BaseWork> leafOperatorToFollowingWork) |
void |
SparkWork.connect(BaseWork a,
BaseWork b,
SparkEdgeProperty edgeProp)
connect adds an edge between a and b.
|
void |
TezWork.connect(BaseWork a,
BaseWork b,
TezEdgeProperty edgeProp)
connect adds an edge between a and b.
|
boolean |
SparkWork.contains(BaseWork w)
Whether the specified BaseWork is a vertex in this graph
|
void |
SparkWork.disconnect(BaseWork a,
BaseWork b)
disconnect removes an edge between a and b.
|
void |
TezWork.disconnect(BaseWork a,
BaseWork b)
disconnect removes an edge between a and b.
|
List<BaseWork> |
SparkWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
List<BaseWork> |
TezWork.getChildren(BaseWork work)
getChildren returns all the nodes with edges leading out of work
|
SparkEdgeProperty |
SparkWork.getEdgeProperty(BaseWork a,
BaseWork b)
returns the edge type connecting work a and b
|
TezEdgeProperty |
TezWork.getEdgeProperty(BaseWork a,
BaseWork b)
returns the edge type connecting work a and b
|
TezEdgeProperty.EdgeType |
TezWork.getEdgeType(BaseWork a,
BaseWork b) |
List<BaseWork> |
SparkWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
List<BaseWork> |
TezWork.getParents(BaseWork work)
getParents returns all the nodes with edges leading into work
|
TezWork.VertexType |
TezWork.getVertexType(BaseWork w) |
void |
SparkWork.remove(BaseWork work)
remove removes a node from the graph and removes all edges with
work as start or end point.
|
void |
TezWork.remove(BaseWork work)
remove removes a node from the graph and removes all edges with
work as start or end point.
|
void |
TezWork.setVertexType(BaseWork w,
TezWork.VertexType incomingVertexType) |
Modifier and Type | Method and Description |
---|---|
void |
SparkWork.addAll(Collection<BaseWork> c)
Add all nodes in the collection without any connections.
|
void |
TezWork.addAll(Collection<BaseWork> c)
add all nodes in the collection without any connections
|
void |
MergeJoinWork.addMergedWork(BaseWork work,
BaseWork connectWork,
Map<Operator<?>,BaseWork> leafOperatorToFollowingWork) |
void |
SparkWork.setCloneToWork(Map<BaseWork,BaseWork> cloneToWork) |
void |
SparkWork.setCloneToWork(Map<BaseWork,BaseWork> cloneToWork) |
Constructor and Description |
---|
BaseExplainVectorization(BaseWork baseWork) |
Copyright © 2021 The Apache Software Foundation. All rights reserved.