PREHOOK: query: -- negative, references twice for result of funcion explain select nkey, nkey + 1 from (select key + 1 as nkey, value from src) a PREHOOK: type: QUERY POSTHOOK: query: -- negative, references twice for result of funcion explain select nkey, nkey + 1 from (select key + 1 as nkey, value from src) a POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (+ (TOK_TABLE_OR_COL key) 1) nkey) (TOK_SELEXPR (TOK_TABLE_OR_COL value))))) a)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL nkey)) (TOK_SELEXPR (+ (TOK_TABLE_OR_COL nkey) 1))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: a:src TableScan alias: src Select Operator expressions: expr: (key + 1) type: double outputColumnNames: _col0 Select Operator expressions: expr: _col0 type: double expr: (_col0 + 1) type: double outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: -- This test query is introduced for HIVE-4968. -- First, we do not convert the join to MapJoin. EXPLAIN SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count PREHOOK: type: QUERY POSTHOOK: query: -- This test query is introduced for HIVE-4968. -- First, we do not convert the join to MapJoin. EXPLAIN SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src1))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_TABLE_OR_COL value))))) tmp1)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)))) tmp2) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src1))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTIONSTAR count) count)))) tmp3))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp2) key) key) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp2) value) value) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp3) count) count)))) tmp4)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) key) key) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) value) value) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) count) count)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL count))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-2 depends on stages: Stage-1 Stage-3 depends on stages: Stage-2 Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: tmp4:tmp3:src1 TableScan alias: src1 Select Operator Group By Operator aggregations: expr: count() bucketGroup: false mode: hash outputColumnNames: _col0 Reduce Output Operator sort order: tag: -1 value expressions: expr: _col0 type: bigint Reduce Operator Tree: Group By Operator aggregations: expr: count(VALUE._col0) bucketGroup: false mode: mergepartial outputColumnNames: _col0 Select Operator expressions: expr: _col0 type: bigint outputColumnNames: _col0 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Stage: Stage-2 Map Reduce Alias -> Map Operator Tree: $INTNAME Reduce Output Operator sort order: tag: 1 value expressions: expr: _col0 type: bigint tmp4:tmp2:tmp1:src1 TableScan alias: src1 Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator sort order: tag: 0 value expressions: expr: _col0 type: string expr: _col1 type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col0} {VALUE._col1} 1 {VALUE._col0} handleSkewJoin: false outputColumnNames: _col0, _col1, _col2 Select Operator expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint outputColumnNames: _col0, _col1, _col2 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Stage: Stage-3 Map Reduce Alias -> Map Operator Tree: #### A masked pattern was here #### Reduce Output Operator key expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint sort order: +++ tag: -1 value expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count PREHOOK: type: QUERY PREHOOK: Input: default@src1 #### A masked pattern was here #### POSTHOOK: query: SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count POSTHOOK: type: QUERY POSTHOOK: Input: default@src1 #### A masked pattern was here #### 25 25 25 25 val_165 25 val_193 25 val_265 25 val_27 25 val_409 25 val_484 25 128 25 146 val_146 25 150 val_150 25 213 val_213 25 224 25 238 val_238 25 255 val_255 25 273 val_273 25 278 val_278 25 311 val_311 25 369 25 401 val_401 25 406 val_406 25 66 val_66 25 98 val_98 25 PREHOOK: query: -- Then, we convert the join to MapJoin. EXPLAIN SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count PREHOOK: type: QUERY POSTHOOK: query: -- Then, we convert the join to MapJoin. EXPLAIN SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src1))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_TABLE_OR_COL value))))) tmp1)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)))) tmp2) (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src1))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTIONSTAR count) count)))) tmp3))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp2) key) key) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp2) value) value) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp3) count) count)))) tmp4)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) key) key) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) value) value) (TOK_SELEXPR (. (TOK_TABLE_OR_COL tmp4) count) count)) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL count))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-6 depends on stages: Stage-1 Stage-3 depends on stages: Stage-6 Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: tmp4:tmp3:src1 TableScan alias: src1 Select Operator Group By Operator aggregations: expr: count() bucketGroup: false mode: hash outputColumnNames: _col0 Reduce Output Operator sort order: tag: -1 value expressions: expr: _col0 type: bigint Reduce Operator Tree: Group By Operator aggregations: expr: count(VALUE._col0) bucketGroup: false mode: mergepartial outputColumnNames: _col0 Select Operator expressions: expr: _col0 type: bigint outputColumnNames: _col0 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat Stage: Stage-6 Map Reduce Local Work Alias -> Map Local Tables: tmp4:tmp2:tmp1:src1 Fetch Operator limit: -1 Alias -> Map Local Operator Tree: tmp4:tmp2:tmp1:src1 TableScan alias: src1 Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 HashTable Sink Operator condition expressions: 0 {_col0} {_col1} 1 {_col0} handleSkewJoin: false keys: 0 [] 1 [] Position of Big Table: 1 Stage: Stage-3 Map Reduce Alias -> Map Operator Tree: $INTNAME Map Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {_col0} {_col1} 1 {_col0} handleSkewJoin: false keys: 0 [] 1 [] outputColumnNames: _col0, _col1, _col2 Position of Big Table: 1 Select Operator expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint outputColumnNames: _col0, _col1, _col2 Reduce Output Operator key expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint sort order: +++ tag: -1 value expressions: expr: _col0 type: string expr: _col1 type: string expr: _col2 type: bigint Local Work: Map Reduce Local Work Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count PREHOOK: type: QUERY PREHOOK: Input: default@src1 #### A masked pattern was here #### POSTHOOK: query: SELECT tmp4.key as key, tmp4.value as value, tmp4.count as count FROM (SELECT tmp2.key as key, tmp2.value as value, tmp3.count as count FROM (SELECT * FROM (SELECT key, value FROM src1) tmp1 ) tmp2 JOIN (SELECT count(*) as count FROM src1) tmp3 ) tmp4 order by key, value, count POSTHOOK: type: QUERY POSTHOOK: Input: default@src1 #### A masked pattern was here #### 25 25 25 25 val_165 25 val_193 25 val_265 25 val_27 25 val_409 25 val_484 25 128 25 146 val_146 25 150 val_150 25 213 val_213 25 224 25 238 val_238 25 255 val_255 25 273 val_273 25 278 val_278 25 311 val_311 25 369 25 401 val_401 25 406 val_406 25 66 val_66 25 98 val_98 25