PREHOOK: query: -- SORT_QUERY_RESULTS -- test cases where the index should not be used automatically CREATE INDEX src_index ON TABLE src(key) as 'COMPACT' WITH DEFERRED REBUILD PREHOOK: type: CREATEINDEX PREHOOK: Input: default@src POSTHOOK: query: -- SORT_QUERY_RESULTS -- test cases where the index should not be used automatically CREATE INDEX src_index ON TABLE src(key) as 'COMPACT' WITH DEFERRED REBUILD POSTHOOK: type: CREATEINDEX POSTHOOK: Input: default@src POSTHOOK: Output: default@default__src_src_index__ PREHOOK: query: ALTER INDEX src_index ON src REBUILD PREHOOK: type: ALTERINDEX_REBUILD PREHOOK: Input: default@src PREHOOK: Output: default@default__src_src_index__ POSTHOOK: query: ALTER INDEX src_index ON src REBUILD POSTHOOK: type: ALTERINDEX_REBUILD POSTHOOK: Input: default@src POSTHOOK: Output: default@default__src_src_index__ POSTHOOK: Lineage: default__src_src_index__._bucketname SIMPLE [(src)src.FieldSchema(name:INPUT__FILE__NAME, type:string, comment:), ] POSTHOOK: Lineage: default__src_src_index__._offsets EXPRESSION [(src)src.FieldSchema(name:BLOCK__OFFSET__INSIDE__FILE, type:bigint, comment:), ] POSTHOOK: Lineage: default__src_src_index__.key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ] PREHOOK: query: -- min size too large (src is less than 5G) EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY POSTHOOK: query: -- min size too large (src is less than 5G) EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: src filterExpr: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 82 val_82 83 val_83 83 val_83 84 val_84 84 val_84 85 val_85 86 val_86 87 val_87 90 val_90 90 val_90 90 val_90 92 val_92 95 val_95 95 val_95 96 val_96 97 val_97 97 val_97 98 val_98 98 val_98 PREHOOK: query: -- max size too small EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY POSTHOOK: query: -- max size too small EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: src filterExpr: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 82 val_82 83 val_83 83 val_83 84 val_84 84 val_84 85 val_85 86 val_86 87 val_87 90 val_90 90 val_90 90 val_90 92 val_92 95 val_95 95 val_95 96 val_96 97 val_97 97 val_97 98 val_98 98 val_98 PREHOOK: query: -- OR predicate not supported by compact indexes EXPLAIN SELECT * FROM src WHERE key < 10 OR key > 480 PREHOOK: type: QUERY POSTHOOK: query: -- OR predicate not supported by compact indexes EXPLAIN SELECT * FROM src WHERE key < 10 OR key > 480 POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: src filterExpr: ((UDFToDouble(key) < 10.0) or (UDFToDouble(key) > 480.0)) (type: boolean) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: ((UDFToDouble(key) < 10.0) or (UDFToDouble(key) > 480.0)) (type: boolean) Statistics: Num rows: 332 Data size: 3527 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 Statistics: Num rows: 332 Data size: 3527 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 332 Data size: 3527 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM src WHERE key < 10 OR key > 480 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM src WHERE key < 10 OR key > 480 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 0 val_0 0 val_0 0 val_0 2 val_2 4 val_4 481 val_481 482 val_482 483 val_483 484 val_484 485 val_485 487 val_487 489 val_489 489 val_489 489 val_489 489 val_489 490 val_490 491 val_491 492 val_492 492 val_492 493 val_493 494 val_494 495 val_495 496 val_496 497 val_497 498 val_498 498 val_498 498 val_498 5 val_5 5 val_5 5 val_5 8 val_8 9 val_9 PREHOOK: query: -- columns are not covered by indexes DROP INDEX src_index on src PREHOOK: type: DROPINDEX PREHOOK: Input: default@src POSTHOOK: query: -- columns are not covered by indexes DROP INDEX src_index on src POSTHOOK: type: DROPINDEX POSTHOOK: Input: default@src PREHOOK: query: CREATE INDEX src_val_index ON TABLE src(value) as 'COMPACT' WITH DEFERRED REBUILD PREHOOK: type: CREATEINDEX PREHOOK: Input: default@src POSTHOOK: query: CREATE INDEX src_val_index ON TABLE src(value) as 'COMPACT' WITH DEFERRED REBUILD POSTHOOK: type: CREATEINDEX POSTHOOK: Input: default@src POSTHOOK: Output: default@default__src_src_val_index__ PREHOOK: query: ALTER INDEX src_val_index ON src REBUILD PREHOOK: type: ALTERINDEX_REBUILD PREHOOK: Input: default@src PREHOOK: Output: default@default__src_src_val_index__ POSTHOOK: query: ALTER INDEX src_val_index ON src REBUILD POSTHOOK: type: ALTERINDEX_REBUILD POSTHOOK: Input: default@src POSTHOOK: Output: default@default__src_src_val_index__ POSTHOOK: Lineage: default__src_src_val_index__._bucketname SIMPLE [(src)src.FieldSchema(name:INPUT__FILE__NAME, type:string, comment:), ] POSTHOOK: Lineage: default__src_src_val_index__._offsets EXPRESSION [(src)src.FieldSchema(name:BLOCK__OFFSET__INSIDE__FILE, type:bigint, comment:), ] POSTHOOK: Lineage: default__src_src_val_index__.value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: src filterExpr: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: ((UDFToDouble(key) > 80.0) and (UDFToDouble(key) < 100.0)) (type: boolean) Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: key (type: string), value (type: string) outputColumnNames: _col0, _col1 Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 55 Data size: 584 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 PREHOOK: type: QUERY PREHOOK: Input: default@src #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM src WHERE key > 80 AND key < 100 POSTHOOK: type: QUERY POSTHOOK: Input: default@src #### A masked pattern was here #### 82 val_82 83 val_83 83 val_83 84 val_84 84 val_84 85 val_85 86 val_86 87 val_87 90 val_90 90 val_90 90 val_90 92 val_92 95 val_95 95 val_95 96 val_96 97 val_97 97 val_97 98 val_98 98 val_98 PREHOOK: query: DROP INDEX src_val_index on src PREHOOK: type: DROPINDEX PREHOOK: Input: default@src POSTHOOK: query: DROP INDEX src_val_index on src POSTHOOK: type: DROPINDEX POSTHOOK: Input: default@src PREHOOK: query: -- required partitions have not been built yet CREATE INDEX src_part_index ON TABLE srcpart(key) as 'COMPACT' WITH DEFERRED REBUILD PREHOOK: type: CREATEINDEX PREHOOK: Input: default@srcpart POSTHOOK: query: -- required partitions have not been built yet CREATE INDEX src_part_index ON TABLE srcpart(key) as 'COMPACT' WITH DEFERRED REBUILD POSTHOOK: type: CREATEINDEX POSTHOOK: Input: default@srcpart POSTHOOK: Output: default@default__srcpart_src_part_index__ PREHOOK: query: ALTER INDEX src_part_index ON srcpart PARTITION (ds='2008-04-08', hr=11) REBUILD PREHOOK: type: ALTERINDEX_REBUILD PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Output: default@default__srcpart_src_part_index__@ds=2008-04-08/hr=11 POSTHOOK: query: ALTER INDEX src_part_index ON srcpart PARTITION (ds='2008-04-08', hr=11) REBUILD POSTHOOK: type: ALTERINDEX_REBUILD POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Output: default@default__srcpart_src_part_index__@ds=2008-04-08/hr=11 POSTHOOK: Lineage: default__srcpart_src_part_index__ PARTITION(ds=2008-04-08,hr=11)._bucketname SIMPLE [(srcpart)srcpart.FieldSchema(name:INPUT__FILE__NAME, type:string, comment:), ] POSTHOOK: Lineage: default__srcpart_src_part_index__ PARTITION(ds=2008-04-08,hr=11)._offsets EXPRESSION [(srcpart)srcpart.FieldSchema(name:BLOCK__OFFSET__INSIDE__FILE, type:bigint, comment:), ] POSTHOOK: Lineage: default__srcpart_src_part_index__ PARTITION(ds=2008-04-08,hr=11).key SIMPLE [(srcpart)srcpart.FieldSchema(name:key, type:string, comment:default), ] PREHOOK: query: EXPLAIN SELECT * FROM srcpart WHERE ds='2008-04-09' AND hr=12 AND key < 10 PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN SELECT * FROM srcpart WHERE ds='2008-04-09' AND hr=12 AND key < 10 POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: srcpart filterExpr: ((12.0 = 12.0) and (UDFToDouble(key) < 10.0)) (type: boolean) Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: ((12.0 = 12.0) and (UDFToDouble(key) < 10.0)) (type: boolean) Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: key (type: string), value (type: string), '2008-04-09' (type: string), '12' (type: string) outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE Column stats: NONE table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM srcpart WHERE ds='2008-04-09' AND hr=12 AND key < 10 PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM srcpart WHERE ds='2008-04-09' AND hr=12 AND key < 10 POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 #### A masked pattern was here #### 0 val_0 2008-04-09 12 0 val_0 2008-04-09 12 0 val_0 2008-04-09 12 2 val_2 2008-04-09 12 4 val_4 2008-04-09 12 5 val_5 2008-04-09 12 5 val_5 2008-04-09 12 5 val_5 2008-04-09 12 8 val_8 2008-04-09 12 9 val_9 2008-04-09 12 PREHOOK: query: DROP INDEX src_part_index on srcpart PREHOOK: type: DROPINDEX PREHOOK: Input: default@srcpart POSTHOOK: query: DROP INDEX src_part_index on srcpart POSTHOOK: type: DROPINDEX POSTHOOK: Input: default@srcpart