PREHOOK: query: -- backward compatible (minimal) explain select * from src limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- backward compatible (minimal) explain select * from src limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 Limit ListSink PREHOOK: query: select * from src limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select * from src limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 238 val_238 86 val_86 311 val_311 27 val_27 165 val_165 409 val_409 255 val_255 278 val_278 98 val_98 484 val_484 PREHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY POSTHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (AND (= (TOK_TABLE_OR_COL ds) '2008-04-08') (= (TOK_TABLE_OR_COL hr) '11'))) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: srcpart Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string outputColumnNames: _col0, _col1, _col2, _col3 Limit ListSink PREHOOK: query: select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### POSTHOOK: query: select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### 238 val_238 2008-04-08 11 86 val_86 2008-04-08 11 311 val_311 2008-04-08 11 27 val_27 2008-04-08 11 165 val_165 2008-04-08 11 409 val_409 2008-04-08 11 255 val_255 2008-04-08 11 278 val_278 2008-04-08 11 98 val_98 2008-04-08 11 484 val_484 2008-04-08 11 PREHOOK: query: -- negative, select expression explain select key from src limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- negative, select expression explain select key from src limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key))) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: key type: string outputColumnNames: _col0 Limit File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: 10 PREHOOK: query: select key from src limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select key from src limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 238 86 311 27 165 409 255 278 98 484 PREHOOK: query: -- negative, filter on non-partition column explain select * from srcpart where key > 100 limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- negative, filter on non-partition column explain select * from srcpart where key > 100 limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (> (TOK_TABLE_OR_COL key) 100)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: srcpart TableScan alias: srcpart Filter Operator predicate: expr: (key > 100) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string outputColumnNames: _col0, _col1, _col2, _col3 Limit File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: 10 PREHOOK: query: select * from srcpart where key > 100 limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: select * from srcpart where key > 100 limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### 238 val_238 2008-04-08 11 311 val_311 2008-04-08 11 165 val_165 2008-04-08 11 409 val_409 2008-04-08 11 255 val_255 2008-04-08 11 278 val_278 2008-04-08 11 484 val_484 2008-04-08 11 265 val_265 2008-04-08 11 193 val_193 2008-04-08 11 401 val_401 2008-04-08 11 PREHOOK: query: -- negative, table sampling explain select * from src TABLESAMPLE (0.25 PERCENT) limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- negative, table sampling explain select * from src TABLESAMPLE (0.25 PERCENT) limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src) (TOK_TABLESPLITSAMPLE TOK_PERCENT 0.25))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 Limit File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Split Sample: src percentage: 0.25 seed number: 0 Stage: Stage-0 Fetch Operator limit: 10 PREHOOK: query: select * from src TABLESAMPLE (0.25 PERCENT) limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select * from src TABLESAMPLE (0.25 PERCENT) limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 238 val_238 86 val_86 311 val_311 27 val_27 165 val_165 409 val_409 255 val_255 278 val_278 98 val_98 484 val_484 PREHOOK: query: -- backward compatible (more) explain select * from src limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- backward compatible (more) explain select * from src limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 Limit ListSink PREHOOK: query: select * from src limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select * from src limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 238 val_238 86 val_86 311 val_311 27 val_27 165 val_165 409 val_409 255 val_255 278 val_278 98 val_98 484 val_484 PREHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY POSTHOOK: query: explain select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (AND (= (TOK_TABLE_OR_COL ds) '2008-04-08') (= (TOK_TABLE_OR_COL hr) '11'))) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: srcpart Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string outputColumnNames: _col0, _col1, _col2, _col3 Limit ListSink PREHOOK: query: select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### POSTHOOK: query: select * from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### 238 val_238 2008-04-08 11 86 val_86 2008-04-08 11 311 val_311 2008-04-08 11 27 val_27 2008-04-08 11 165 val_165 2008-04-08 11 409 val_409 2008-04-08 11 255 val_255 2008-04-08 11 278 val_278 2008-04-08 11 98 val_98 2008-04-08 11 484 val_484 2008-04-08 11 PREHOOK: query: -- select expression explain select cast(key as int) * 10, upper(value) from src limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- select expression explain select cast(key as int) * 10, upper(value) from src limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (* (TOK_FUNCTION TOK_INT (TOK_TABLE_OR_COL key)) 10)) (TOK_SELEXPR (TOK_FUNCTION upper (TOK_TABLE_OR_COL value)))) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: src Select Operator expressions: expr: (UDFToInteger(key) * 10) type: int expr: upper(value) type: string outputColumnNames: _col0, _col1 Limit ListSink PREHOOK: query: select cast(key as int) * 10, upper(value) from src limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select cast(key as int) * 10, upper(value) from src limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 2380 VAL_238 860 VAL_86 3110 VAL_311 270 VAL_27 1650 VAL_165 4090 VAL_409 2550 VAL_255 2780 VAL_278 980 VAL_98 4840 VAL_484 PREHOOK: query: -- filter on non-partition column explain select key from src where key < 100 limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- filter on non-partition column explain select key from src where key < 100 limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key))) (TOK_WHERE (< (TOK_TABLE_OR_COL key) 100)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: src Filter Operator predicate: expr: (key < 100) type: boolean Select Operator expressions: expr: key type: string outputColumnNames: _col0 Limit ListSink PREHOOK: query: select key from src where key < 100 limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select key from src where key < 100 limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 86 27 98 66 37 15 82 17 0 57 PREHOOK: query: -- select expr for partitioned table explain select key from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- select expr for partitioned table explain select key from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key))) (TOK_WHERE (AND (= (TOK_TABLE_OR_COL ds) '2008-04-08') (= (TOK_TABLE_OR_COL hr) '11'))) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: srcpart Select Operator expressions: expr: key type: string outputColumnNames: _col0 Limit ListSink PREHOOK: query: select key from srcpart where ds='2008-04-08' AND hr='11' limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### POSTHOOK: query: select key from srcpart where ds='2008-04-08' AND hr='11' limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 #### A masked pattern was here #### 238 86 311 27 165 409 255 278 98 484 PREHOOK: query: -- virtual columns explain select *, BLOCK__OFFSET__INSIDE__FILE from src where key < 10 limit 10 PREHOOK: type: QUERY POSTHOOK: query: -- virtual columns explain select *, BLOCK__OFFSET__INSIDE__FILE from src where key < 10 limit 10 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))) (TOK_WHERE (< (TOK_TABLE_OR_COL key) 10)) (TOK_LIMIT 10))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 10 Processor Tree: TableScan alias: src Filter Operator predicate: expr: (key < 10) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2 Limit ListSink PREHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from src where key < 100 limit 10 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from src where key < 100 limit 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 86 val_86 12 27 val_27 34 98 val_98 92 66 val_66 198 37 val_37 328 15 val_15 386 82 val_82 396 17 val_17 910 0 val_0 968 57 val_57 1024 PREHOOK: query: -- virtual columns on partitioned table explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart where key < 10 limit 30 PREHOOK: type: QUERY POSTHOOK: query: -- virtual columns on partitioned table explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart where key < 10 limit 30 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))) (TOK_WHERE (< (TOK_TABLE_OR_COL key) 10)) (TOK_LIMIT 30))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: 30 Processor Tree: TableScan alias: srcpart Filter Operator predicate: expr: (key < 10) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2, _col3, _col4 Limit ListSink PREHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart where key < 10 limit 30 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart where key < 10 limit 30 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### 0 val_0 2008-04-08 11 968 4 val_4 2008-04-08 11 1218 8 val_8 2008-04-08 11 1916 0 val_0 2008-04-08 11 2088 0 val_0 2008-04-08 11 2632 5 val_5 2008-04-08 11 3060 5 val_5 2008-04-08 11 3864 2 val_2 2008-04-08 11 4004 5 val_5 2008-04-08 11 4540 9 val_9 2008-04-08 11 5398 0 val_0 2008-04-08 12 968 4 val_4 2008-04-08 12 1218 8 val_8 2008-04-08 12 1916 0 val_0 2008-04-08 12 2088 0 val_0 2008-04-08 12 2632 5 val_5 2008-04-08 12 3060 5 val_5 2008-04-08 12 3864 2 val_2 2008-04-08 12 4004 5 val_5 2008-04-08 12 4540 9 val_9 2008-04-08 12 5398 0 val_0 2008-04-09 11 968 4 val_4 2008-04-09 11 1218 8 val_8 2008-04-09 11 1916 0 val_0 2008-04-09 11 2088 0 val_0 2008-04-09 11 2632 5 val_5 2008-04-09 11 3060 5 val_5 2008-04-09 11 3864 2 val_2 2008-04-09 11 4004 5 val_5 2008-04-09 11 4540 9 val_9 2008-04-09 11 5398 PREHOOK: query: -- bucket sampling explain select *, BLOCK__OFFSET__INSIDE__FILE from src TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) PREHOOK: type: QUERY POSTHOOK: query: -- bucket sampling explain select *, BLOCK__OFFSET__INSIDE__FILE from src TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src) (TOK_TABLEBUCKETSAMPLE 1 40 (TOK_TABLE_OR_COL key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: TableScan alias: src Filter Operator predicate: expr: (((hash(key) & 2147483647) % 40) = 0) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2 ListSink PREHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from src TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from src TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 484 val_484 102 286 val_286 1404 187 val_187 1416 187 val_187 2492 77 val_77 2622 187 val_187 4516 448 val_448 5636 PREHOOK: query: explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) PREHOOK: type: QUERY POSTHOOK: query: explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart) (TOK_TABLEBUCKETSAMPLE 1 40 (TOK_TABLE_OR_COL key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: TableScan alias: srcpart Filter Operator predicate: expr: (((hash(key) & 2147483647) % 40) = 0) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2, _col3, _col4 ListSink PREHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (BUCKET 1 OUT OF 40 ON key) POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### 484 val_484 2008-04-08 11 102 286 val_286 2008-04-08 11 1404 187 val_187 2008-04-08 11 1416 187 val_187 2008-04-08 11 2492 77 val_77 2008-04-08 11 2622 187 val_187 2008-04-08 11 4516 448 val_448 2008-04-08 11 5636 484 val_484 2008-04-08 12 102 286 val_286 2008-04-08 12 1404 187 val_187 2008-04-08 12 1416 187 val_187 2008-04-08 12 2492 77 val_77 2008-04-08 12 2622 187 val_187 2008-04-08 12 4516 448 val_448 2008-04-08 12 5636 484 val_484 2008-04-09 11 102 286 val_286 2008-04-09 11 1404 187 val_187 2008-04-09 11 1416 187 val_187 2008-04-09 11 2492 77 val_77 2008-04-09 11 2622 187 val_187 2008-04-09 11 4516 448 val_448 2008-04-09 11 5636 484 val_484 2008-04-09 12 102 286 val_286 2008-04-09 12 1404 187 val_187 2008-04-09 12 1416 187 val_187 2008-04-09 12 2492 77 val_77 2008-04-09 12 2622 187 val_187 2008-04-09 12 4516 448 val_448 2008-04-09 12 5636 PREHOOK: query: -- split sampling explain select * from src TABLESAMPLE (0.25 PERCENT) PREHOOK: type: QUERY POSTHOOK: query: -- split sampling explain select * from src TABLESAMPLE (0.25 PERCENT) POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src) (TOK_TABLESPLITSAMPLE TOK_PERCENT 0.25))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 ListSink PREHOOK: query: select * from src TABLESAMPLE (0.25 PERCENT) PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: select * from src TABLESAMPLE (0.25 PERCENT) POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 238 val_238 86 val_86 PREHOOK: query: explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (0.25 PERCENT) PREHOOK: type: QUERY POSTHOOK: query: explain select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (0.25 PERCENT) POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart) (TOK_TABLESPLITSAMPLE TOK_PERCENT 0.25))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: TableScan alias: srcpart Select Operator expressions: expr: key type: string expr: value type: string expr: ds type: string expr: hr type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2, _col3, _col4 ListSink PREHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (0.25 PERCENT) PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: select *, BLOCK__OFFSET__INSIDE__FILE from srcpart TABLESAMPLE (0.25 PERCENT) POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### 238 val_238 2008-04-08 11 0 86 val_86 2008-04-08 11 12 238 val_238 2008-04-08 12 0 86 val_86 2008-04-08 12 12 238 val_238 2008-04-09 11 0 86 val_86 2008-04-09 11 12 238 val_238 2008-04-09 12 0 86 val_86 2008-04-09 12 12 PREHOOK: query: -- non deterministic func explain select key, value, BLOCK__OFFSET__INSIDE__FILE from srcpart where ds="2008-04-09" AND rand() > 1 PREHOOK: type: QUERY POSTHOOK: query: -- non deterministic func explain select key, value, BLOCK__OFFSET__INSIDE__FILE from srcpart where ds="2008-04-09" AND rand() > 1 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcpart))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_TABLE_OR_COL value)) (TOK_SELEXPR (TOK_TABLE_OR_COL BLOCK__OFFSET__INSIDE__FILE))) (TOK_WHERE (AND (= (TOK_TABLE_OR_COL ds) "2008-04-09") (> (TOK_FUNCTION rand) 1))))) STAGE DEPENDENCIES: Stage-0 is a root stage STAGE PLANS: Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: TableScan alias: srcpart Filter Operator predicate: expr: (rand() > 1) type: boolean Select Operator expressions: expr: key type: string expr: value type: string expr: BLOCK__OFFSET__INSIDE__FILE type: bigint outputColumnNames: _col0, _col1, _col2 ListSink PREHOOK: query: select key, value, BLOCK__OFFSET__INSIDE__FILE from srcpart where ds="2008-04-09" AND rand() > 1 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: select key, value, BLOCK__OFFSET__INSIDE__FILE from srcpart where ds="2008-04-09" AND rand() > 1 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### PREHOOK: query: -- negative, groupby explain select key, count(value) from src group by key PREHOOK: type: QUERY POSTHOOK: query: -- negative, groupby explain select key, count(value) from src group by key POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_FUNCTION count (TOK_TABLE_OR_COL value)))) (TOK_GROUPBY (TOK_TABLE_OR_COL key)))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: key, value Group By Operator aggregations: expr: count(value) bucketGroup: false keys: expr: key type: string mode: hash outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: string sort order: + Map-reduce partition columns: expr: _col0 type: string tag: -1 value expressions: expr: _col1 type: bigint Reduce Operator Tree: Group By Operator aggregations: expr: count(VALUE._col0) bucketGroup: false keys: expr: KEY._col0 type: string mode: mergepartial outputColumnNames: _col0, _col1 Select Operator expressions: expr: _col0 type: string expr: _col1 type: bigint outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: -- negative, distinct explain select distinct key, value from src PREHOOK: type: QUERY POSTHOOK: query: -- negative, distinct explain select distinct key, value from src POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECTDI (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: key, value Group By Operator bucketGroup: false keys: expr: key type: string expr: value type: string mode: hash outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: string expr: _col1 type: string sort order: ++ Map-reduce partition columns: expr: _col0 type: string expr: _col1 type: string tag: -1 Reduce Operator Tree: Group By Operator bucketGroup: false keys: expr: KEY._col0 type: string expr: KEY._col1 type: string mode: mergepartial outputColumnNames: _col0, _col1 Select Operator expressions: expr: _col0 type: string expr: _col1 type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: -- negative, CTAS explain create table srcx as select distinct key, value from src PREHOOK: type: CREATETABLE_AS_SELECT POSTHOOK: query: -- negative, CTAS explain create table srcx as select distinct key, value from src POSTHOOK: type: CREATETABLE_AS_SELECT ABSTRACT SYNTAX TREE: (TOK_CREATETABLE (TOK_TABNAME srcx) TOK_LIKETABLE (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECTDI (TOK_SELEXPR (TOK_TABLE_OR_COL key)) (TOK_SELEXPR (TOK_TABLE_OR_COL value)))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 Stage-3 depends on stages: Stage-0 Stage-2 depends on stages: Stage-3 STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: key, value Group By Operator bucketGroup: false keys: expr: key type: string expr: value type: string mode: hash outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: string expr: _col1 type: string sort order: ++ Map-reduce partition columns: expr: _col0 type: string expr: _col1 type: string tag: -1 Reduce Operator Tree: Group By Operator bucketGroup: false keys: expr: KEY._col0 type: string expr: KEY._col1 type: string mode: mergepartial outputColumnNames: _col0, _col1 Select Operator expressions: expr: _col0 type: string expr: _col1 type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat name: default.srcx Stage: Stage-0 Move Operator files: hdfs directory: true #### A masked pattern was here #### Stage: Stage-3 Create Table Operator: Create Table columns: key string, value string if not exists: false input format: org.apache.hadoop.mapred.TextInputFormat # buckets: -1 output format: org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat name: srcx isExternal: false Stage: Stage-2 Stats-Aggr Operator PREHOOK: query: -- negative, analyze explain analyze table src compute statistics PREHOOK: type: QUERY POSTHOOK: query: -- negative, analyze explain analyze table src compute statistics POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_ANALYZE (TOK_TAB (TOK_TABNAME src))) STAGE DEPENDENCIES: Stage-0 is a root stage Stage-1 depends on stages: Stage-0 STAGE PLANS: Stage: Stage-0 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Stage: Stage-1 Stats-Aggr Operator PREHOOK: query: -- negative, subq explain select a.* from (select * from src) a PREHOOK: type: QUERY POSTHOOK: query: -- negative, subq explain select a.* from (select * from src) a POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)))) a)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME a)))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: a:src TableScan alias: src Select Operator expressions: expr: key type: string expr: value type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: -- negative, join explain select * from src join src src2 on src.key=src2.key PREHOOK: type: QUERY POSTHOOK: query: -- negative, join explain select * from src join src src2 on src.key=src2.key POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME src)) (TOK_TABREF (TOK_TABNAME src) src2) (= (. (TOK_TABLE_OR_COL src) key) (. (TOK_TABLE_OR_COL src2) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Reduce Output Operator key expressions: expr: key type: string sort order: + Map-reduce partition columns: expr: key type: string tag: 0 value expressions: expr: key type: string expr: value type: string src2 TableScan alias: src2 Reduce Output Operator key expressions: expr: key type: string sort order: + Map-reduce partition columns: expr: key type: string tag: 1 value expressions: expr: key type: string expr: value type: string Reduce Operator Tree: Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {VALUE._col0} {VALUE._col1} 1 {VALUE._col0} {VALUE._col1} handleSkewJoin: false outputColumnNames: _col0, _col1, _col4, _col5 Select Operator expressions: expr: _col0 type: string expr: _col1 type: string expr: _col4 type: string expr: _col5 type: string outputColumnNames: _col0, _col1, _col2, _col3 File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: -1