PREHOOK: query: -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23) -- List bucketing query logic test case. We simulate the directory structure by DML here. -- Test condition: -- 1. where clause has multiple skewed columns and non-skewed columns -- 3. where clause has a few operators -- Test focus: -- 1. query works for on partition level. -- A table can mix up non-skewed partition and skewed partition -- Even for skewed partition, it can have different skewed information. -- Test result: -- 1. pruner only pick up right directory -- 2. query result is right -- create a skewed table create table fact_daily (key String, value String) partitioned by (ds String, hr String) PREHOOK: type: CREATETABLE PREHOOK: Output: database:default PREHOOK: Output: default@fact_daily POSTHOOK: query: -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.23) -- List bucketing query logic test case. We simulate the directory structure by DML here. -- Test condition: -- 1. where clause has multiple skewed columns and non-skewed columns -- 3. where clause has a few operators -- Test focus: -- 1. query works for on partition level. -- A table can mix up non-skewed partition and skewed partition -- Even for skewed partition, it can have different skewed information. -- Test result: -- 1. pruner only pick up right directory -- 2. query result is right -- create a skewed table create table fact_daily (key String, value String) partitioned by (ds String, hr String) POSTHOOK: type: CREATETABLE POSTHOOK: Output: database:default POSTHOOK: Output: default@fact_daily PREHOOK: query: -- partition no skew insert overwrite table fact_daily partition (ds = '1', hr = '1') select key, value from src PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@fact_daily@ds=1/hr=1 POSTHOOK: query: -- partition no skew insert overwrite table fact_daily partition (ds = '1', hr = '1') select key, value from src POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@fact_daily@ds=1/hr=1 POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=1).key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=1).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='1') PREHOOK: type: DESCTABLE PREHOOK: Input: default@fact_daily POSTHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='1') POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@fact_daily # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [1, 1] Database: default Table: fact_daily #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE true numFiles 1 numRows 500 rawDataSize 5312 totalSize 5812 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: serialization.format 1 PREHOOK: query: -- partition. skewed value is 484/238 alter table fact_daily skewed by (key, value) on (('484','val_484'),('238','val_238')) stored as DIRECTORIES PREHOOK: type: ALTERTABLE_SKEWED PREHOOK: Input: default@fact_daily PREHOOK: Output: default@fact_daily POSTHOOK: query: -- partition. skewed value is 484/238 alter table fact_daily skewed by (key, value) on (('484','val_484'),('238','val_238')) stored as DIRECTORIES POSTHOOK: type: ALTERTABLE_SKEWED POSTHOOK: Input: default@fact_daily POSTHOOK: Output: default@fact_daily PREHOOK: query: insert overwrite table fact_daily partition (ds = '1', hr = '2') select key, value from src PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@fact_daily@ds=1/hr=2 POSTHOOK: query: insert overwrite table fact_daily partition (ds = '1', hr = '2') select key, value from src POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@fact_daily@ds=1/hr=2 POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=2).key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=2).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='2') PREHOOK: type: DESCTABLE PREHOOK: Input: default@fact_daily POSTHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='2') POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@fact_daily # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [1, 2] Database: default Table: fact_daily #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE true numFiles 3 numRows 500 rawDataSize 5312 totalSize 5812 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Stored As SubDirectories: Yes Skewed Columns: [key, value] Skewed Values: [[484, val_484], [238, val_238]] #### A masked pattern was here #### Skewed Value to Truncated Path: {[484, val_484]=/fact_daily/ds=1/hr=2/key=484/value=val_484, [238, val_238]=/fact_daily/ds=1/hr=2/key=238/value=val_238} Storage Desc Params: serialization.format 1 PREHOOK: query: -- another partition. skewed value is 327 alter table fact_daily skewed by (key, value) on (('327','val_327')) stored as DIRECTORIES PREHOOK: type: ALTERTABLE_SKEWED PREHOOK: Input: default@fact_daily PREHOOK: Output: default@fact_daily POSTHOOK: query: -- another partition. skewed value is 327 alter table fact_daily skewed by (key, value) on (('327','val_327')) stored as DIRECTORIES POSTHOOK: type: ALTERTABLE_SKEWED POSTHOOK: Input: default@fact_daily POSTHOOK: Output: default@fact_daily PREHOOK: query: insert overwrite table fact_daily partition (ds = '1', hr = '3') select key, value from src PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@fact_daily@ds=1/hr=3 POSTHOOK: query: insert overwrite table fact_daily partition (ds = '1', hr = '3') select key, value from src POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@fact_daily@ds=1/hr=3 POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=3).key SIMPLE [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: fact_daily PARTITION(ds=1,hr=3).value SIMPLE [(src)src.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='3') PREHOOK: type: DESCTABLE PREHOOK: Input: default@fact_daily POSTHOOK: query: describe formatted fact_daily PARTITION (ds = '1', hr='3') POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@fact_daily # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [1, 3] Database: default Table: fact_daily #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE true numFiles 2 numRows 500 rawDataSize 5312 totalSize 5812 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe InputFormat: org.apache.hadoop.mapred.TextInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Stored As SubDirectories: Yes Skewed Columns: [key, value] Skewed Values: [[327, val_327]] #### A masked pattern was here #### Skewed Value to Truncated Path: {[327, val_327]=/fact_daily/ds=1/hr=3/key=327/value=val_327} Storage Desc Params: serialization.format 1 PREHOOK: query: -- query non-skewed partition explain extended select * from fact_daily where ds = '1' and hr='1' and key='145' PREHOOK: type: QUERY POSTHOOK: query: -- query non-skewed partition explain extended select * from fact_daily where ds = '1' and hr='1' and key='145' POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_TABREF TOK_TABNAME fact_daily TOK_INSERT TOK_DESTINATION TOK_DIR TOK_TMP_FILE TOK_SELECT TOK_SELEXPR TOK_ALLCOLREF TOK_WHERE and and = TOK_TABLE_OR_COL ds '1' = TOK_TABLE_OR_COL hr '1' = TOK_TABLE_OR_COL key '145' STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: fact_daily Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE GatherStats: false Filter Operator isSamplingPred: false predicate: (key = '145') (type: boolean) Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: '145' (type: string), value (type: string), '1' (type: string), '1' (type: string) outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1,_col2,_col3 columns.types string:string:string:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe TotalFiles: 1 GatherStats: false MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: hr=1 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat partition values: ds 1 hr 1 properties: COLUMN_STATS_ACCURATE true bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily numFiles 1 numRows 500 partition_columns ds/hr partition_columns.types string:string rawDataSize 5312 serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily partition_columns ds/hr partition_columns.types string:string serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.fact_daily name: default.fact_daily Truncated Path -> Alias: /fact_daily/ds=1/hr=1 [fact_daily] Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: select * from fact_daily where ds = '1' and hr='1' and key='145' PREHOOK: type: QUERY PREHOOK: Input: default@fact_daily PREHOOK: Input: default@fact_daily@ds=1/hr=1 #### A masked pattern was here #### POSTHOOK: query: select * from fact_daily where ds = '1' and hr='1' and key='145' POSTHOOK: type: QUERY POSTHOOK: Input: default@fact_daily POSTHOOK: Input: default@fact_daily@ds=1/hr=1 #### A masked pattern was here #### 145 val_145 1 1 PREHOOK: query: explain extended select count(*) from fact_daily where ds = '1' and hr='1' PREHOOK: type: QUERY POSTHOOK: query: explain extended select count(*) from fact_daily where ds = '1' and hr='1' POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_TABREF TOK_TABNAME fact_daily TOK_INSERT TOK_DESTINATION TOK_DIR TOK_TMP_FILE TOK_SELECT TOK_SELEXPR TOK_FUNCTIONSTAR count TOK_WHERE and = TOK_TABLE_OR_COL ds '1' = TOK_TABLE_OR_COL hr '1' STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: fact_daily Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE GatherStats: false Select Operator Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count() mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE Reduce Output Operator sort order: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE tag: -1 value expressions: _col0 (type: bigint) auto parallelism: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: hr=1 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat partition values: ds 1 hr 1 properties: COLUMN_STATS_ACCURATE true bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily numFiles 1 numRows 500 partition_columns ds/hr partition_columns.types string:string rawDataSize 5312 serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily partition_columns ds/hr partition_columns.types string:string serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.fact_daily name: default.fact_daily Truncated Path -> Alias: /fact_daily/ds=1/hr=1 [$hdt$_0:$hdt$_0:fact_daily] Needs Tagging: false Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0) mode: mergepartial outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: _col0 (type: bigint) outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0 columns.types bigint escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: select count(*) from fact_daily where ds = '1' and hr='1' PREHOOK: type: QUERY PREHOOK: Input: default@fact_daily PREHOOK: Input: default@fact_daily@ds=1/hr=1 #### A masked pattern was here #### POSTHOOK: query: select count(*) from fact_daily where ds = '1' and hr='1' POSTHOOK: type: QUERY POSTHOOK: Input: default@fact_daily POSTHOOK: Input: default@fact_daily@ds=1/hr=1 #### A masked pattern was here #### 500 PREHOOK: query: -- query skewed partition explain extended SELECT * FROM fact_daily WHERE ds='1' and hr='2' and (key='484' and value='val_484') PREHOOK: type: QUERY POSTHOOK: query: -- query skewed partition explain extended SELECT * FROM fact_daily WHERE ds='1' and hr='2' and (key='484' and value='val_484') POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_TABREF TOK_TABNAME fact_daily TOK_INSERT TOK_DESTINATION TOK_DIR TOK_TMP_FILE TOK_SELECT TOK_SELEXPR TOK_ALLCOLREF TOK_WHERE and and = TOK_TABLE_OR_COL ds '1' = TOK_TABLE_OR_COL hr '2' and = TOK_TABLE_OR_COL key '484' = TOK_TABLE_OR_COL value 'val_484' STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: fact_daily Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE GatherStats: false Filter Operator isSamplingPred: false predicate: ((key = '484') and (value = 'val_484')) (type: boolean) Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: '484' (type: string), 'val_484' (type: string), '1' (type: string), '2' (type: string) outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1,_col2,_col3 columns.types string:string:string:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe TotalFiles: 1 GatherStats: false MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: value=val_484 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat partition values: ds 1 hr 2 properties: COLUMN_STATS_ACCURATE true bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily numFiles 3 numRows 500 partition_columns ds/hr partition_columns.types string:string rawDataSize 5312 serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily partition_columns ds/hr partition_columns.types string:string serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.fact_daily name: default.fact_daily Truncated Path -> Alias: /fact_daily/ds=1/hr=2/key=484/value=val_484 [fact_daily] Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM fact_daily WHERE ds='1' and hr='2' and (key='484' and value='val_484') PREHOOK: type: QUERY PREHOOK: Input: default@fact_daily PREHOOK: Input: default@fact_daily@ds=1/hr=2 #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM fact_daily WHERE ds='1' and hr='2' and (key='484' and value='val_484') POSTHOOK: type: QUERY POSTHOOK: Input: default@fact_daily POSTHOOK: Input: default@fact_daily@ds=1/hr=2 #### A masked pattern was here #### 484 val_484 1 2 PREHOOK: query: -- query another skewed partition explain extended SELECT * FROM fact_daily WHERE ds='1' and hr='3' and (key='327' and value='val_327') PREHOOK: type: QUERY POSTHOOK: query: -- query another skewed partition explain extended SELECT * FROM fact_daily WHERE ds='1' and hr='3' and (key='327' and value='val_327') POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: TOK_QUERY TOK_FROM TOK_TABREF TOK_TABNAME fact_daily TOK_INSERT TOK_DESTINATION TOK_DIR TOK_TMP_FILE TOK_SELECT TOK_SELEXPR TOK_ALLCOLREF TOK_WHERE and and = TOK_TABLE_OR_COL ds '1' = TOK_TABLE_OR_COL hr '3' and = TOK_TABLE_OR_COL key '327' = TOK_TABLE_OR_COL value 'val_327' STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 depends on stages: Stage-1 STAGE PLANS: Stage: Stage-1 Map Reduce Map Operator Tree: TableScan alias: fact_daily Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE GatherStats: false Filter Operator isSamplingPred: false predicate: ((key = '327') and (value = 'val_327')) (type: boolean) Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: '327' (type: string), 'val_327' (type: string), '1' (type: string), '3' (type: string) outputColumnNames: _col0, _col1, _col2, _col3 Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 Statistics: Num rows: 125 Data size: 1328 Basic stats: COMPLETE Column stats: NONE #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1,_col2,_col3 columns.types string:string:string:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe TotalFiles: 1 GatherStats: false MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: value=val_327 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat partition values: ds 1 hr 3 properties: COLUMN_STATS_ACCURATE true bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily numFiles 2 numRows 500 partition_columns ds/hr partition_columns.types string:string rawDataSize 5312 serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.comments columns.types string:string #### A masked pattern was here #### name default.fact_daily partition_columns ds/hr partition_columns.types string:string serialization.ddl struct fact_daily { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.fact_daily name: default.fact_daily Truncated Path -> Alias: /fact_daily/ds=1/hr=3/key=327/value=val_327 [fact_daily] Stage: Stage-0 Fetch Operator limit: -1 Processor Tree: ListSink PREHOOK: query: SELECT * FROM fact_daily WHERE ds='1' and hr='3' and (key='327' and value='val_327') PREHOOK: type: QUERY PREHOOK: Input: default@fact_daily PREHOOK: Input: default@fact_daily@ds=1/hr=3 #### A masked pattern was here #### POSTHOOK: query: SELECT * FROM fact_daily WHERE ds='1' and hr='3' and (key='327' and value='val_327') POSTHOOK: type: QUERY POSTHOOK: Input: default@fact_daily POSTHOOK: Input: default@fact_daily@ds=1/hr=3 #### A masked pattern was here #### 327 val_327 1 3 327 val_327 1 3 327 val_327 1 3