Package org.apache.calcite.sql2rel
Class RelDecorrelator
- java.lang.Object
-
- org.apache.calcite.sql2rel.RelDecorrelator
-
- All Implemented Interfaces:
ReflectiveVisitor
public class RelDecorrelator extends java.lang.Object implements ReflectiveVisitor
RelDecorrelator replaces all correlated expressions (corExp) in a relational expression (RelNode) tree with non-correlated expressions that are produced from joining the RelNode that produces the corExp with the RelNode that references it.TODO:
- replace
CorelMap
constructor parameter with a RelNode - make
currentRel
immutable (would require a fresh RelDecorrelator for each node being decorrelated) - make fields of
CorelMap
immutable - make sub-class rules static, and have them create their own de-correlator
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private class
RelDecorrelator.AdjustProjectForCountAggregateRule
Planner rule that adjusts projects when counts are added.(package private) static class
RelDecorrelator.CorDef
A correlation and a field.private static class
RelDecorrelator.CorelMap
A map of the locations ofLogicalCorrelate
in a tree ofRelNode
s.private static class
RelDecorrelator.CorelMapBuilder
Builds aRelDecorrelator.CorelMap
.(package private) static class
RelDecorrelator.CorRef
A unique reference to a correlation field.private static class
RelDecorrelator.DecorrelateRexShuttle
Shuttle that decorrelates.(package private) static class
RelDecorrelator.Frame
Frame describing the relational expression after decorrelation and where to find the output fields and correlation variables among its output fields.private class
RelDecorrelator.RemoveCorrelationForScalarAggregateRule
Planner rule that removes correlations for scalar aggregates.private class
RelDecorrelator.RemoveCorrelationForScalarProjectRule
Planner rule that removes correlations for scalar projects.private class
RelDecorrelator.RemoveCorrelationRexShuttle
Shuttle that removes correlations.private class
RelDecorrelator.RemoveSingleAggregateRule
Rule to remove single_value rel.
-
Field Summary
Fields Modifier and Type Field Description private RelDecorrelator.CorelMap
cm
private Context
context
private RelNode
currentRel
private ReflectUtil.MethodDispatcher<RelDecorrelator.Frame>
dispatcher
private java.util.HashSet<LogicalCorrelate>
generatedCorRels
private java.util.Map<RelNode,RelDecorrelator.Frame>
map
Built during decorrelation, of rel to all the newly created correlated variables in its output, and to map old input positions to new input positions.private RelBuilder
relBuilder
private static org.slf4j.Logger
SQL2REL_LOGGER
-
Constructor Summary
Constructors Modifier Constructor Description private
RelDecorrelator(RelDecorrelator.CorelMap cm, Context context, RelBuilder relBuilder)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description private RelNode
aggregateCorrelatorOutput(Correlate correlate, LogicalProject project, java.util.Set<java.lang.Integer> isCount)
(package private) static boolean
allLessThan(java.util.Collection<java.lang.Integer> integers, int limit, Litmus ret)
private boolean
checkCorVars(LogicalCorrelate correlate, LogicalProject project, LogicalFilter filter, java.util.List<RexFieldAccess> correlatedJoinKeys)
Checks whether the correlations in projRel and filter are related to the correlated variables provided by corRel.private Function2<RelNode,RelNode,java.lang.Void>
createCopyHook()
private HepPlanner
createPlanner(HepProgram program)
private RelNode
createProjectWithAdditionalExprs(RelNode input, java.util.List<Pair<RexNode,java.lang.String>> additionalExprs)
Projects allinput
output fields plus the additional expressions.private RelNode
createValueGenerator(java.lang.Iterable<RelDecorrelator.CorRef> correlations, int valueGenFieldOffset, java.util.SortedMap<RelDecorrelator.CorDef,java.lang.Integer> corDefOutputs)
Create RelNode tree that produces a list of correlated variables.private RelNode
decorrelate(RelNode root)
protected RexNode
decorrelateExpr(RelNode currentRel, java.util.Map<RelNode,RelDecorrelator.Frame> map, RelDecorrelator.CorelMap cm, RexNode exp)
private RelDecorrelator.Frame
decorrelateInputWithValueGenerator(RelNode rel, RelDecorrelator.Frame frame)
static RelNode
decorrelateQuery(RelNode rootRel)
Deprecated.static RelNode
decorrelateQuery(RelNode rootRel, RelBuilder relBuilder)
Decorrelates a query.RelDecorrelator.Frame
decorrelateRel(Sort rel)
Rewrite Sort.RelDecorrelator.Frame
decorrelateRel(Values rel)
Rewrites aValues
.RelDecorrelator.Frame
decorrelateRel(LogicalAggregate rel)
Rewrites aLogicalAggregate
.RelDecorrelator.Frame
decorrelateRel(LogicalCorrelate rel)
Rewrite Correlate into a left outer join.RelDecorrelator.Frame
decorrelateRel(LogicalFilter rel)
Rewrite LogicalFilter.RelDecorrelator.Frame
decorrelateRel(LogicalJoin rel)
Rewrite LogicalJoin.RelDecorrelator.Frame
decorrelateRel(LogicalProject rel)
Rewrite LogicalProject.RelDecorrelator.Frame
decorrelateRel(RelNode rel)
Fallback if none of the otherdecorrelateRel
methods match.private void
findCorrelationEquivalent(RelDecorrelator.CorRef correlation, RexNode e)
Finds aRexInputRef
that is equivalent to aRelDecorrelator.CorRef
, and if found, throws aUtil.FoundOne
.private RelNode
getCorRel(RelDecorrelator.CorRef corVar)
private RelDecorrelator.Frame
getFrame(RelNode r, boolean safe)
RelDecorrelator.Frame
getInvoke(RelNode r, RelNode parent)
private static RexInputRef
getNewForOldInputRef(RelNode currentRel, java.util.Map<RelNode,RelDecorrelator.Frame> map, RexInputRef oldInputRef)
private boolean
has(java.util.Collection<RelDecorrelator.CorDef> corDefs, RelDecorrelator.CorRef corr)
Returns whether aCorrelationId
is satisfied by at least one of a collection ofRelDecorrelator.CorDef
s.private boolean
hasAll(java.util.Collection<RelDecorrelator.CorRef> corRefs, java.util.Collection<RelDecorrelator.CorDef> corDefs)
Returns whether all of a collection ofRelDecorrelator.CorRef
s are satisfied by at least one of a collection ofRelDecorrelator.CorDef
s.(package private) static java.util.Map<java.lang.Integer,java.lang.Integer>
identityMap(int count)
private boolean
isWidening(RelDataType type, RelDataType type1)
Returns whether one type is just a widening of another.private RelDecorrelator.Frame
maybeAddValueGenerator(RelNode rel, RelDecorrelator.Frame frame)
Adds a value generator to satisfy the correlating variables used by a relational expression, if those variables are not already provided by its input.private static RexLiteral
projectedLiteral(RelNode rel, int i)
Returns a literal output field, or null if it is not literal.private RelNode
projectJoinOutputWithNullability(LogicalJoin join, LogicalProject project, int nullIndicatorPos)
Pulls project above the join from its RHS input.private boolean
references(RexNode e, RelDecorrelator.CorRef correlation)
(package private) RelDecorrelator.Frame
register(RelNode rel, RelNode newRel, java.util.Map<java.lang.Integer,java.lang.Integer> oldToNewOutputs, java.util.SortedMap<RelDecorrelator.CorDef,java.lang.Integer> corDefOutputs)
Registers a relational expression and the relational expression it became after decorrelation.private RelBuilderFactory
relBuilderFactory()
protected RexNode
removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator)
protected RexNode
removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, java.util.Set<java.lang.Integer> isCount)
protected RexNode
removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, RexInputRef nullIndicator)
RelNode
removeCorrelationViaRule(RelNode root)
private void
removeCorVarFromTree(LogicalCorrelate correlate)
Remove correlated variables from the tree at root corRelprivate void
setCurrent(RelNode root, LogicalCorrelate corRel)
private static RelNode
stripHep(RelNode rel)
-
-
-
Field Detail
-
SQL2REL_LOGGER
private static final org.slf4j.Logger SQL2REL_LOGGER
-
relBuilder
private final RelBuilder relBuilder
-
cm
private RelDecorrelator.CorelMap cm
-
dispatcher
private final ReflectUtil.MethodDispatcher<RelDecorrelator.Frame> dispatcher
-
currentRel
private RelNode currentRel
-
context
private final Context context
-
map
private final java.util.Map<RelNode,RelDecorrelator.Frame> map
Built during decorrelation, of rel to all the newly created correlated variables in its output, and to map old input positions to new input positions. This is from the view point of the parent rel of a new rel.
-
generatedCorRels
private final java.util.HashSet<LogicalCorrelate> generatedCorRels
-
-
Constructor Detail
-
RelDecorrelator
private RelDecorrelator(RelDecorrelator.CorelMap cm, Context context, RelBuilder relBuilder)
-
-
Method Detail
-
decorrelateQuery
public static RelNode decorrelateQuery(RelNode rootRel, RelBuilder relBuilder)
Decorrelates a query.This is the main entry point to
RelDecorrelator
.- Parameters:
rootRel
- Root node of the queryrelBuilder
- Builder for relational expressions- Returns:
- Equivalent query with all
LogicalCorrelate
instances removed
-
setCurrent
private void setCurrent(RelNode root, LogicalCorrelate corRel)
-
relBuilderFactory
private RelBuilderFactory relBuilderFactory()
-
createPlanner
private HepPlanner createPlanner(HepProgram program)
-
decorrelateExpr
protected RexNode decorrelateExpr(RelNode currentRel, java.util.Map<RelNode,RelDecorrelator.Frame> map, RelDecorrelator.CorelMap cm, RexNode exp)
-
removeCorrelationExpr
protected RexNode removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator)
-
removeCorrelationExpr
protected RexNode removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, RexInputRef nullIndicator)
-
removeCorrelationExpr
protected RexNode removeCorrelationExpr(RexNode exp, boolean projectPulledAboveLeftCorrelator, java.util.Set<java.lang.Integer> isCount)
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(RelNode rel)
Fallback if none of the otherdecorrelateRel
methods match.
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(Sort rel)
Rewrite Sort.- Parameters:
rel
- Sort to be rewritten
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(Values rel)
Rewrites aValues
.- Parameters:
rel
- Values to be rewritten
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(LogicalAggregate rel)
Rewrites aLogicalAggregate
.- Parameters:
rel
- Aggregate to rewrite
-
getInvoke
public RelDecorrelator.Frame getInvoke(RelNode r, RelNode parent)
-
projectedLiteral
private static RexLiteral projectedLiteral(RelNode rel, int i)
Returns a literal output field, or null if it is not literal.
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(LogicalProject rel)
Rewrite LogicalProject.- Parameters:
rel
- the project rel to rewrite
-
createValueGenerator
private RelNode createValueGenerator(java.lang.Iterable<RelDecorrelator.CorRef> correlations, int valueGenFieldOffset, java.util.SortedMap<RelDecorrelator.CorDef,java.lang.Integer> corDefOutputs)
Create RelNode tree that produces a list of correlated variables.- Parameters:
correlations
- correlated variables to generatevalueGenFieldOffset
- offset in the output that generated columns will startcorDefOutputs
- output positions for the correlated variables generated- Returns:
- RelNode the root of the resultant RelNode tree
-
getFrame
private RelDecorrelator.Frame getFrame(RelNode r, boolean safe)
-
getCorRel
private RelNode getCorRel(RelDecorrelator.CorRef corVar)
-
maybeAddValueGenerator
private RelDecorrelator.Frame maybeAddValueGenerator(RelNode rel, RelDecorrelator.Frame frame)
Adds a value generator to satisfy the correlating variables used by a relational expression, if those variables are not already provided by its input.
-
hasAll
private boolean hasAll(java.util.Collection<RelDecorrelator.CorRef> corRefs, java.util.Collection<RelDecorrelator.CorDef> corDefs)
Returns whether all of a collection ofRelDecorrelator.CorRef
s are satisfied by at least one of a collection ofRelDecorrelator.CorDef
s.
-
has
private boolean has(java.util.Collection<RelDecorrelator.CorDef> corDefs, RelDecorrelator.CorRef corr)
Returns whether aCorrelationId
is satisfied by at least one of a collection ofRelDecorrelator.CorDef
s.
-
decorrelateInputWithValueGenerator
private RelDecorrelator.Frame decorrelateInputWithValueGenerator(RelNode rel, RelDecorrelator.Frame frame)
-
findCorrelationEquivalent
private void findCorrelationEquivalent(RelDecorrelator.CorRef correlation, RexNode e) throws Util.FoundOne
Finds aRexInputRef
that is equivalent to aRelDecorrelator.CorRef
, and if found, throws aUtil.FoundOne
.- Throws:
Util.FoundOne
-
references
private boolean references(RexNode e, RelDecorrelator.CorRef correlation)
-
isWidening
private boolean isWidening(RelDataType type, RelDataType type1)
Returns whether one type is just a widening of another.For example:
VARCHAR(10)
is a widening ofVARCHAR(5)
.VARCHAR(10)
is a widening ofVARCHAR(10) NOT NULL
.
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(LogicalFilter rel)
Rewrite LogicalFilter.- Parameters:
rel
- the filter rel to rewrite
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(LogicalCorrelate rel)
Rewrite Correlate into a left outer join.- Parameters:
rel
- Correlator
-
decorrelateRel
public RelDecorrelator.Frame decorrelateRel(LogicalJoin rel)
Rewrite LogicalJoin.- Parameters:
rel
- Join
-
getNewForOldInputRef
private static RexInputRef getNewForOldInputRef(RelNode currentRel, java.util.Map<RelNode,RelDecorrelator.Frame> map, RexInputRef oldInputRef)
-
projectJoinOutputWithNullability
private RelNode projectJoinOutputWithNullability(LogicalJoin join, LogicalProject project, int nullIndicatorPos)
Pulls project above the join from its RHS input. Enforces nullability for join output.- Parameters:
join
- Joinproject
- Original project as the right-hand input of the joinnullIndicatorPos
- Position of null indicator- Returns:
- the subtree with the new Project at the root
-
aggregateCorrelatorOutput
private RelNode aggregateCorrelatorOutput(Correlate correlate, LogicalProject project, java.util.Set<java.lang.Integer> isCount)
- Parameters:
correlate
- Correlateproject
- the original project as the RHS input of the joinisCount
- Positions which are calls to theCOUNT
aggregation function- Returns:
- the subtree with the new Project at the root
-
checkCorVars
private boolean checkCorVars(LogicalCorrelate correlate, LogicalProject project, LogicalFilter filter, java.util.List<RexFieldAccess> correlatedJoinKeys)
Checks whether the correlations in projRel and filter are related to the correlated variables provided by corRel.- Parameters:
correlate
- Correlateproject
- The original Project as the RHS input of the joinfilter
- FiltercorrelatedJoinKeys
- Correlated join keys- Returns:
- true if filter and proj only references corVar provided by corRel
-
removeCorVarFromTree
private void removeCorVarFromTree(LogicalCorrelate correlate)
Remove correlated variables from the tree at root corRel- Parameters:
correlate
- Correlate
-
createProjectWithAdditionalExprs
private RelNode createProjectWithAdditionalExprs(RelNode input, java.util.List<Pair<RexNode,java.lang.String>> additionalExprs)
Projects allinput
output fields plus the additional expressions.- Parameters:
input
- Input relational expressionadditionalExprs
- Additional expressions and names- Returns:
- the new Project
-
identityMap
static java.util.Map<java.lang.Integer,java.lang.Integer> identityMap(int count)
-
register
RelDecorrelator.Frame register(RelNode rel, RelNode newRel, java.util.Map<java.lang.Integer,java.lang.Integer> oldToNewOutputs, java.util.SortedMap<RelDecorrelator.CorDef,java.lang.Integer> corDefOutputs)
Registers a relational expression and the relational expression it became after decorrelation.
-
allLessThan
static boolean allLessThan(java.util.Collection<java.lang.Integer> integers, int limit, Litmus ret)
-
-