public class ReduceSinkDeDuplicationUtils extends Object
Constructor and Description |
---|
ReduceSinkDeDuplicationUtils() |
Modifier and Type | Method and Description |
---|---|
protected static boolean |
aggressiveDedup(ReduceSinkOperator cRS,
ReduceSinkOperator pRS,
ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx) |
protected static Integer |
checkNumReducer(int creduce,
int preduce)
If number of reducers for RS is -1, the RS can have any number of reducers.
|
protected static Integer |
checkOrder(boolean checkStrictEquality,
String corder,
String porder,
String cNullOrder,
String pNullOrder) |
static boolean |
merge(ReduceSinkOperator cRS,
JoinOperator pJoin,
int minReducer) |
static boolean |
merge(ReduceSinkOperator cRS,
ReduceSinkOperator pRS,
int minReducer)
Current RSDedup remove/replace child RS.
|
protected static Integer |
sameKeys(List<ExprNodeDesc> cexprs,
List<ExprNodeDesc> pexprs,
Operator<?> child,
Operator<?> parent) |
static boolean |
strictMerge(ReduceSinkOperator cRS,
List<ReduceSinkOperator> pRSs) |
static boolean |
strictMerge(ReduceSinkOperator cRS,
ReduceSinkOperator pRS)
This is a more strict version of the merge check, where:
- cRS and pRS should have exactly the same keys in the same positions, and
- cRS and pRS should have exactly the same partition columns in the same positions, and
- cRS and pRS should have exactly the same bucket columns in the same positions, and
- cRS and pRS should sort in the same direction
|
public static boolean merge(ReduceSinkOperator cRS, JoinOperator pJoin, int minReducer) throws SemanticException
SemanticException
public static boolean merge(ReduceSinkOperator cRS, ReduceSinkOperator pRS, int minReducer) throws SemanticException
SemanticException
public static boolean strictMerge(ReduceSinkOperator cRS, ReduceSinkOperator pRS) throws SemanticException
SemanticException
public static boolean strictMerge(ReduceSinkOperator cRS, List<ReduceSinkOperator> pRSs) throws SemanticException
SemanticException
protected static Integer sameKeys(List<ExprNodeDesc> cexprs, List<ExprNodeDesc> pexprs, Operator<?> child, Operator<?> parent) throws SemanticException
SemanticException
protected static Integer checkOrder(boolean checkStrictEquality, String corder, String porder, String cNullOrder, String pNullOrder)
protected static Integer checkNumReducer(int creduce, int preduce)
protected static boolean aggressiveDedup(ReduceSinkOperator cRS, ReduceSinkOperator pRS, ReduceSinkDeDuplication.ReduceSinkDeduplicateProcCtx dedupCtx) throws SemanticException
SemanticException
Copyright © 2022 The Apache Software Foundation. All rights reserved.