Class SubstitutionVisitor
- java.lang.Object
-
- org.apache.calcite.plan.SubstitutionVisitor
-
- Direct Known Subclasses:
MaterializedViewSubstitutionVisitor
public class SubstitutionVisitor extends java.lang.Object
Substitutes part of a tree of relational expressions with another tree.The call
new SubstitutionVisitor(target, query).go(replacement))
will returnquery
with every occurrence oftarget
replaced byreplacement
.The following example shows how
SubstitutionVisitor
can be used for materialized view recognition.- query = SELECT a, c FROM t WHERE x = 5 AND b = 4
- target = SELECT a, b, c FROM t WHERE x = 5
- replacement = SELECT * FROM mv
- result = SELECT a, c FROM mv WHERE b = 4
Note that
result
uses the materialized view tablemv
and a simplified conditionb = 4
.Uses a bottom-up matching algorithm. Nodes do not need to be identical. At each level, returns the residue.
The inputs must only include the core relational operators:
LogicalTableScan
,LogicalFilter
,LogicalProject
,LogicalJoin
,LogicalUnion
,LogicalAggregate
.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
SubstitutionVisitor.AbstractUnifyRule
Abstract base class for implementingSubstitutionVisitor.UnifyRule
.private static class
SubstitutionVisitor.AggregateOnProjectToAggregateUnifyRule
Implementation ofSubstitutionVisitor.UnifyRule
that matches aMutableAggregate
on aMutableProject
query to anMutableAggregate
target.private static class
SubstitutionVisitor.AggregateToAggregateUnifyRule
Implementation ofSubstitutionVisitor.UnifyRule
that matches aLogicalAggregate
to aLogicalAggregate
, provided that they have the same child.private static class
SubstitutionVisitor.AnyOperand
Operand to aSubstitutionVisitor.UnifyRule
that matches a relational expression of a given type.static class
SubstitutionVisitor.FilterOnProjectRule
Rule that converts aLogicalFilter
on top of aLogicalProject
into a trivial filter (on a boolean column).private static class
SubstitutionVisitor.FilterToFilterUnifyRule
Implementation ofSubstitutionVisitor.UnifyRule
that matches aMutableFilter
.private static class
SubstitutionVisitor.FilterToProjectUnifyRule
private static class
SubstitutionVisitor.InternalOperand
Operand to aSubstitutionVisitor.UnifyRule
that matches a relational expression of a given type.protected static class
SubstitutionVisitor.MatchFailed
Exception thrown to exit a matcher.protected static class
SubstitutionVisitor.Operand
Operand to aSubstitutionVisitor.UnifyRule
.private static class
SubstitutionVisitor.ProjectToFilterUnifyRule
private static class
SubstitutionVisitor.ProjectToProjectUnifyRule
Implementation ofSubstitutionVisitor.UnifyRule
that matchesLogicalProject
.private static class
SubstitutionVisitor.QueryOperand
Operand that assigns a particular relational expression to a variable.(package private) static class
SubstitutionVisitor.Replacement
Represents a replacement action: before → after.private static class
SubstitutionVisitor.ScanToProjectUnifyRule
Implementation ofSubstitutionVisitor.UnifyRule
that matchesLogicalTableScan
.private static class
SubstitutionVisitor.SlotCounter
Visitor that counts how manySubstitutionVisitor.QueryOperand
andSubstitutionVisitor.TargetOperand
in an operand tree.private static class
SubstitutionVisitor.TargetOperand
Operand that checks that a relational expression matches the corresponding relational expression that was passed to aSubstitutionVisitor.QueryOperand
.private static class
SubstitutionVisitor.TrivialRule
Implementation ofSubstitutionVisitor.UnifyRule
that matches if the query is already equal to the target.protected static class
SubstitutionVisitor.UnifyResult
Result of an application of aSubstitutionVisitor.UnifyRule
indicating that the rule successfully matchedquery
againsttarget
and generated aresult
that is equivalent toquery
and containstarget
.protected static class
SubstitutionVisitor.UnifyRule
Rule that attempts to match a query relational expression against a target relational expression.protected class
SubstitutionVisitor.UnifyRuleCall
Arguments to an application of aSubstitutionVisitor.UnifyRule
.
-
Field Summary
Fields Modifier and Type Field Description private RelOptCluster
cluster
private static boolean
DEBUG
protected static com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule>
DEFAULT_RULES
(package private) com.google.common.collect.Multimap<MutableRel,MutableRel>
equivalents
private static org.slf4j.Logger
LOGGER
private Holder
query
(package private) java.util.List<MutableRel>
queryLeaves
Nodes inquery
that have no children.protected RelBuilder
relBuilder
Factory for a builder for relational expressions.(package private) java.util.Map<MutableRel,MutableRel>
replacementMap
private java.util.Map<Pair<java.lang.Class,java.lang.Class>,java.util.List<SubstitutionVisitor.UnifyRule>>
ruleMap
private com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule>
rules
private RexSimplify
simplify
protected MutableRel[]
slots
Workspace while rule is being matched.private MutableRel
target
(package private) java.util.List<MutableRel>
targetLeaves
Nodes intarget
that have no children.
-
Constructor Summary
Constructors Constructor Description SubstitutionVisitor(RelNode target_, RelNode query_)
Creates a SubstitutionVisitor with the default rule set.SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules)
Creates a SubstitutionVisitor with the default logical builder.SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules, RelBuilderFactory relBuilderFactory)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private java.util.List<SubstitutionVisitor.UnifyRule>
applicableRules(MutableRel query, MutableRel target)
private SubstitutionVisitor.UnifyResult
apply(SubstitutionVisitor.UnifyRule rule, MutableRel query, MutableRel target)
private static RexNode
canonizeNode(RexBuilder rexBuilder, RexNode condition)
Reorders some of the operands in this expression so structural comparison, i.e., based on string representation, can be more precise.static boolean
equalType(java.lang.String desc0, MutableRel rel0, java.lang.String desc1, MutableRel rel1, Litmus litmus)
Returns whether two relational expressions have the same row-type.protected static RexShuttle
getRexShuttle(MutableProject target)
Builds a shuttle that stores a list of expressions, and can map incoming expressions to references to them.static SqlAggFunction
getRollup(SqlAggFunction aggregation)
private java.util.List<java.util.List<SubstitutionVisitor.Replacement>>
go(MutableRel replacement)
Substitutes the query with replacement whenever possible but meanwhile keeps track of all the substitutions and their original rel before replacement, so that in later processing stage, the replacement can be recovered individually to produce a list of all possible rels with substitution in different places.java.util.List<RelNode>
go(RelNode replacement_)
Returns a list of all possible rels that result from substituting the matched RelNode with the replacement RelNode within the query.RelNode
go0(RelNode replacement_)
private static boolean
isEquivalent(RexBuilder rexBuilder, RexNode condition, RexNode target)
protected boolean
isWeaker(MutableRel rel0, MutableRel rel)
Returns if one rel is weaker than another.private SubstitutionVisitor.UnifyResult
matchRecurse(MutableRel target)
static boolean
mayBeSatisfiable(RexNode e)
Returns whether a boolean expression ever returns true.private static boolean
mightMatch(SubstitutionVisitor.UnifyRule rule, java.lang.Class queryClass, java.lang.Class targetClass)
static MutableAggregate
permute(MutableAggregate aggregate, MutableRel input, Mapping mapping)
private static void
redoReplacement(java.util.List<SubstitutionVisitor.Replacement> replacement)
(package private) void
register(MutableRel result, MutableRel query)
static SubstitutionVisitor.Replacement
replace(MutableRel query, MutableRel find, MutableRel replace)
Within a relational expressionquery
, replaces occurrences offind
withreplace
.private static SubstitutionVisitor.Replacement
replaceRecurse(MutableRel query, MutableRel find, MutableRel replace)
private static void
reverseSubstitute(RelBuilder relBuilder, Holder query, java.util.List<java.util.List<SubstitutionVisitor.Replacement>> matches, java.util.List<RelNode> sub, int replaceCount, int maxCount)
static RexNode
splitFilter(RexSimplify simplify, RexNode condition, RexNode target)
Maps a condition onto a target.private static RexNode
splitOr(RexBuilder rexBuilder, RexNode condition, RexNode target)
private static void
undoReplacement(java.util.List<SubstitutionVisitor.Replacement> replacement)
static MutableRel
unifyAggregates(MutableAggregate query, MutableAggregate target)
-
-
-
Field Detail
-
DEBUG
private static final boolean DEBUG
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
DEFAULT_RULES
protected static final com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> DEFAULT_RULES
-
relBuilder
protected final RelBuilder relBuilder
Factory for a builder for relational expressions.
-
rules
private final com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules
-
ruleMap
private final java.util.Map<Pair<java.lang.Class,java.lang.Class>,java.util.List<SubstitutionVisitor.UnifyRule>> ruleMap
-
cluster
private final RelOptCluster cluster
-
simplify
private final RexSimplify simplify
-
query
private final Holder query
-
target
private final MutableRel target
-
targetLeaves
final java.util.List<MutableRel> targetLeaves
Nodes intarget
that have no children.
-
queryLeaves
final java.util.List<MutableRel> queryLeaves
Nodes inquery
that have no children.
-
replacementMap
final java.util.Map<MutableRel,MutableRel> replacementMap
-
equivalents
final com.google.common.collect.Multimap<MutableRel,MutableRel> equivalents
-
slots
protected final MutableRel[] slots
Workspace while rule is being matched. Careful, re-entrant! Assumes no rule needs more than 2 slots.
-
-
Constructor Detail
-
SubstitutionVisitor
public SubstitutionVisitor(RelNode target_, RelNode query_)
Creates a SubstitutionVisitor with the default rule set.
-
SubstitutionVisitor
public SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules)
Creates a SubstitutionVisitor with the default logical builder.
-
SubstitutionVisitor
public SubstitutionVisitor(RelNode target_, RelNode query_, com.google.common.collect.ImmutableList<SubstitutionVisitor.UnifyRule> rules, RelBuilderFactory relBuilderFactory)
-
-
Method Detail
-
register
void register(MutableRel result, MutableRel query)
-
splitFilter
public static RexNode splitFilter(RexSimplify simplify, RexNode condition, RexNode target)
Maps a condition onto a target.If condition is stronger than target, returns the residue. If it is equal to target, returns the expression that evaluates to the constant
true
. If it is weaker than target, returnsnull
.The terms satisfy the relation
condition = target AND residue
and
residue
must be as weak as possible.Example #1: condition stronger than target
- condition: x = 1 AND y = 2
- target: x = 1
- residue: y = 2
Note that residue
x > 0 AND y = 2
would also satisfy the relationcondition = target AND residue
but is stronger than necessary, so we prefery = 2
.Example #2: target weaker than condition (valid, but not currently implemented)
- condition: x = 1
- target: x = 1 OR z = 3
- residue: x = 1
Example #3: condition and target are equivalent
- condition: x = 1 AND y = 2
- target: y = 2 AND x = 1
- residue: TRUE
Example #4: condition weaker than target
- condition: x = 1
- target: x = 1 AND y = 2
- residue: null (i.e. no match)
There are many other possible examples. It amounts to solving whether
condition AND NOT target
can ever evaluate to true, and therefore is a form of the NP-complete Satisfiability problem.
-
canonizeNode
private static RexNode canonizeNode(RexBuilder rexBuilder, RexNode condition)
Reorders some of the operands in this expression so structural comparison, i.e., based on string representation, can be more precise.
-
splitOr
private static RexNode splitOr(RexBuilder rexBuilder, RexNode condition, RexNode target)
-
isEquivalent
private static boolean isEquivalent(RexBuilder rexBuilder, RexNode condition, RexNode target)
-
mayBeSatisfiable
public static boolean mayBeSatisfiable(RexNode e)
Returns whether a boolean expression ever returns true.This method may give false positives. For instance, it will say that
x = 5 AND x > 10
is satisfiable, because at present it cannot prove that it is not.
-
go
public java.util.List<RelNode> go(RelNode replacement_)
Returns a list of all possible rels that result from substituting the matched RelNode with the replacement RelNode within the query.For example, the substitution result of A join B, while A and B are both a qualified match for replacement R, is R join B, R join R, A join R.
-
go
private java.util.List<java.util.List<SubstitutionVisitor.Replacement>> go(MutableRel replacement)
Substitutes the query with replacement whenever possible but meanwhile keeps track of all the substitutions and their original rel before replacement, so that in later processing stage, the replacement can be recovered individually to produce a list of all possible rels with substitution in different places.
-
replace
public static SubstitutionVisitor.Replacement replace(MutableRel query, MutableRel find, MutableRel replace)
Within a relational expressionquery
, replaces occurrences offind
withreplace
.Assumes relational expressions (and their descendants) are not null. Does not handle cycles.
-
replaceRecurse
private static SubstitutionVisitor.Replacement replaceRecurse(MutableRel query, MutableRel find, MutableRel replace)
-
undoReplacement
private static void undoReplacement(java.util.List<SubstitutionVisitor.Replacement> replacement)
-
redoReplacement
private static void redoReplacement(java.util.List<SubstitutionVisitor.Replacement> replacement)
-
reverseSubstitute
private static void reverseSubstitute(RelBuilder relBuilder, Holder query, java.util.List<java.util.List<SubstitutionVisitor.Replacement>> matches, java.util.List<RelNode> sub, int replaceCount, int maxCount)
-
matchRecurse
private SubstitutionVisitor.UnifyResult matchRecurse(MutableRel target)
-
apply
private SubstitutionVisitor.UnifyResult apply(SubstitutionVisitor.UnifyRule rule, MutableRel query, MutableRel target)
-
applicableRules
private java.util.List<SubstitutionVisitor.UnifyRule> applicableRules(MutableRel query, MutableRel target)
-
mightMatch
private static boolean mightMatch(SubstitutionVisitor.UnifyRule rule, java.lang.Class queryClass, java.lang.Class targetClass)
-
permute
public static MutableAggregate permute(MutableAggregate aggregate, MutableRel input, Mapping mapping)
-
unifyAggregates
public static MutableRel unifyAggregates(MutableAggregate query, MutableAggregate target)
-
getRollup
public static SqlAggFunction getRollup(SqlAggFunction aggregation)
-
getRexShuttle
protected static RexShuttle getRexShuttle(MutableProject target)
Builds a shuttle that stores a list of expressions, and can map incoming expressions to references to them.
-
isWeaker
protected boolean isWeaker(MutableRel rel0, MutableRel rel)
Returns if one rel is weaker than another.
-
equalType
public static boolean equalType(java.lang.String desc0, MutableRel rel0, java.lang.String desc1, MutableRel rel1, Litmus litmus)
Returns whether two relational expressions have the same row-type.
-
-