PREHOOK: query: -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.20,0.20S) -- This test uses mapred.max.split.size/mapred.max.split.size for controlling -- number of input splits, which is not effective in hive 0.20. -- stats_partscan_1_23.q is the same test with this but has different result. -- test analyze table ... compute statistics partialscan -- 1. prepare data CREATE table analyze_srcpart_partial_scan (key STRING, value STRING) partitioned by (ds string, hr string) stored as rcfile PREHOOK: type: CREATETABLE PREHOOK: Output: database:default POSTHOOK: query: -- INCLUDE_HADOOP_MAJOR_VERSIONS(0.20,0.20S) -- This test uses mapred.max.split.size/mapred.max.split.size for controlling -- number of input splits, which is not effective in hive 0.20. -- stats_partscan_1_23.q is the same test with this but has different result. -- test analyze table ... compute statistics partialscan -- 1. prepare data CREATE table analyze_srcpart_partial_scan (key STRING, value STRING) partitioned by (ds string, hr string) stored as rcfile POSTHOOK: type: CREATETABLE POSTHOOK: Output: database:default POSTHOOK: Output: default@analyze_srcpart_partial_scan PREHOOK: query: insert overwrite table analyze_srcpart_partial_scan partition (ds, hr) select * from srcpart where ds is not null PREHOOK: type: QUERY PREHOOK: Input: default@srcpart PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 PREHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 PREHOOK: Output: default@analyze_srcpart_partial_scan POSTHOOK: query: insert overwrite table analyze_srcpart_partial_scan partition (ds, hr) select * from srcpart where ds is not null POSTHOOK: type: QUERY POSTHOOK: Input: default@srcpart POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-08/hr=12 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=11 POSTHOOK: Input: default@srcpart@ds=2008-04-09/hr=12 POSTHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=11 POSTHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=12 POSTHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-09/hr=11 POSTHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-09/hr=12 POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-08,hr=11).key SIMPLE [(srcpart)srcpart.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-08,hr=11).value SIMPLE [(srcpart)srcpart.FieldSchema(name:value, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-08,hr=12).key SIMPLE [(srcpart)srcpart.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-08,hr=12).value SIMPLE [(srcpart)srcpart.FieldSchema(name:value, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-09,hr=11).key SIMPLE [(srcpart)srcpart.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-09,hr=11).value SIMPLE [(srcpart)srcpart.FieldSchema(name:value, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-09,hr=12).key SIMPLE [(srcpart)srcpart.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: analyze_srcpart_partial_scan PARTITION(ds=2008-04-09,hr=12).value SIMPLE [(srcpart)srcpart.FieldSchema(name:value, type:string, comment:default), ] PREHOOK: query: describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) PREHOOK: type: DESCTABLE PREHOOK: Input: default@analyze_srcpart_partial_scan POSTHOOK: query: describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@analyze_srcpart_partial_scan # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [2008-04-08, 11] Database: default Table: analyze_srcpart_partial_scan #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE false numFiles 1 numRows -1 rawDataSize -1 totalSize 5293 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe InputFormat: org.apache.hadoop.hive.ql.io.RCFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.RCFileOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: serialization.format 1 PREHOOK: query: -- 2. partialscan explain analyze table analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) compute statistics partialscan PREHOOK: type: QUERY POSTHOOK: query: -- 2. partialscan explain analyze table analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) compute statistics partialscan POSTHOOK: type: QUERY STAGE DEPENDENCIES: Stage-2 is a root stage Stage-1 depends on stages: Stage-2 STAGE PLANS: Stage: Stage-2 Partial Scan Statistics Stage: Stage-1 Stats-Aggr Operator PREHOOK: query: analyze table analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) compute statistics partialscan PREHOOK: type: QUERY PREHOOK: Input: default@analyze_srcpart_partial_scan PREHOOK: Input: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=11 PREHOOK: Output: default@analyze_srcpart_partial_scan PREHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=11 POSTHOOK: query: analyze table analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) compute statistics partialscan POSTHOOK: type: QUERY POSTHOOK: Input: default@analyze_srcpart_partial_scan POSTHOOK: Input: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=11 POSTHOOK: Output: default@analyze_srcpart_partial_scan POSTHOOK: Output: default@analyze_srcpart_partial_scan@ds=2008-04-08/hr=11 PREHOOK: query: -- 3. confirm result describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) PREHOOK: type: DESCTABLE PREHOOK: Input: default@analyze_srcpart_partial_scan POSTHOOK: query: -- 3. confirm result describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-08',hr=11) POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@analyze_srcpart_partial_scan # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [2008-04-08, 11] Database: default Table: analyze_srcpart_partial_scan #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE true numFiles 1 numRows 500 rawDataSize 4812 totalSize 5293 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe InputFormat: org.apache.hadoop.hive.ql.io.RCFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.RCFileOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: serialization.format 1 PREHOOK: query: describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-09',hr=11) PREHOOK: type: DESCTABLE PREHOOK: Input: default@analyze_srcpart_partial_scan POSTHOOK: query: describe formatted analyze_srcpart_partial_scan PARTITION(ds='2008-04-09',hr=11) POSTHOOK: type: DESCTABLE POSTHOOK: Input: default@analyze_srcpart_partial_scan # col_name data_type comment key string value string # Partition Information # col_name data_type comment ds string hr string # Detailed Partition Information Partition Value: [2008-04-09, 11] Database: default Table: analyze_srcpart_partial_scan #### A masked pattern was here #### Protect Mode: None #### A masked pattern was here #### Partition Parameters: COLUMN_STATS_ACCURATE false numFiles 1 numRows -1 rawDataSize -1 totalSize 5293 #### A masked pattern was here #### # Storage Information SerDe Library: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe InputFormat: org.apache.hadoop.hive.ql.io.RCFileInputFormat OutputFormat: org.apache.hadoop.hive.ql.io.RCFileOutputFormat Compressed: No Num Buckets: -1 Bucket Columns: [] Sort Columns: [] Storage Desc Params: serialization.format 1 PREHOOK: query: drop table analyze_srcpart_partial_scan PREHOOK: type: DROPTABLE PREHOOK: Input: default@analyze_srcpart_partial_scan PREHOOK: Output: default@analyze_srcpart_partial_scan POSTHOOK: query: drop table analyze_srcpart_partial_scan POSTHOOK: type: DROPTABLE POSTHOOK: Input: default@analyze_srcpart_partial_scan POSTHOOK: Output: default@analyze_srcpart_partial_scan