PREHOOK: query: DESCRIBE FUNCTION explode PREHOOK: type: DESCFUNCTION POSTHOOK: query: DESCRIBE FUNCTION explode POSTHOOK: type: DESCFUNCTION explode(a) - separates the elements of array a into multiple rows, or the elements of a map into multiple rows and columns PREHOOK: query: DESCRIBE FUNCTION EXTENDED explode PREHOOK: type: DESCFUNCTION POSTHOOK: query: DESCRIBE FUNCTION EXTENDED explode POSTHOOK: type: DESCFUNCTION explode(a) - separates the elements of array a into multiple rows, or the elements of a map into multiple rows and columns PREHOOK: query: EXPLAIN EXTENDED SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3 PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION explode (TOK_FUNCTION array 1 2 3)) myCol)) (TOK_LIMIT 3))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src GatherStats: false Select Operator expressions: expr: array(1,2,3) type: array outputColumnNames: _col0 UDTF Operator function name: explode Limit File Output Operator compressed: false GlobalTableId: 0 directory: file:/tmp/amarsri/hive_2011-08-23_03-07-02_928_1250194056464136663/-ext-10001 NumFilesPerFileSink: 1 Stats Publishing Key Prefix: file:/tmp/amarsri/hive_2011-08-23_03-07-02_928_1250194056464136663/-ext-10001/ table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns col columns.types int escape.delim \ serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Needs Tagging: false Path -> Alias: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src [src] Path -> Partition: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src Partition base file name: src input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.src name: default.src Stage: Stage-0 Fetch Operator limit: 3 PREHOOK: query: EXPLAIN EXTENDED SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3) a GROUP BY a.myCol PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3) a GROUP BY a.myCol POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION explode (TOK_FUNCTION array 1 2 3)) myCol)) (TOK_LIMIT 3))) a)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) myCol)) (TOK_SELEXPR (TOK_FUNCTION count 1))) (TOK_GROUPBY (. (TOK_TABLE_OR_COL a) myCol)))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-2 depends on stages: Stage-1 Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: a:src TableScan alias: src GatherStats: false Select Operator expressions: expr: array(1,2,3) type: array outputColumnNames: _col0 UDTF Operator function name: explode Limit Reduce Output Operator sort order: tag: -1 value expressions: expr: col type: int Needs Tagging: false Path -> Alias: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src [a:src] Path -> Partition: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src Partition base file name: src input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.src name: default.src Reduce Operator Tree: Extract Limit Select Operator expressions: expr: _col0 type: int outputColumnNames: _col0 Group By Operator aggregations: expr: count(1) bucketGroup: false keys: expr: _col0 type: int mode: hash outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 directory: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-mr-10002 NumFilesPerFileSink: 1 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1 columns.types int,bigint escape.delim \ TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-2 Map Reduce Alias -> Map Operator Tree: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-mr-10002 Reduce Output Operator key expressions: expr: _col0 type: int sort order: + Map-reduce partition columns: expr: _col0 type: int tag: -1 value expressions: expr: _col1 type: bigint Needs Tagging: false Path -> Alias: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-mr-10002 [file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-mr-10002] Path -> Partition: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-mr-10002 Partition base file name: -mr-10002 input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1 columns.types int,bigint escape.delim \ input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1 columns.types int,bigint escape.delim \ Reduce Operator Tree: Group By Operator aggregations: expr: count(VALUE._col0) bucketGroup: false keys: expr: KEY._col0 type: int mode: mergepartial outputColumnNames: _col0, _col1 Select Operator expressions: expr: _col0 type: int expr: _col1 type: bigint outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 0 directory: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-ext-10001 NumFilesPerFileSink: 1 Stats Publishing Key Prefix: file:/tmp/amarsri/hive_2011-08-23_03-07-02_968_9198802378494217934/-ext-10001/ table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:bigint escape.delim \ serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3 PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-03_015_4726265824126226982/-mr-10000 POSTHOOK: query: SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3 POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-03_015_4726265824126226982/-mr-10000 1 2 3 PREHOOK: query: SELECT explode(array(1,2,3)) AS (myCol) FROM src LIMIT 3 PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-05_538_8498964664323960073/-mr-10000 POSTHOOK: query: SELECT explode(array(1,2,3)) AS (myCol) FROM src LIMIT 3 POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-05_538_8498964664323960073/-mr-10000 1 2 3 PREHOOK: query: SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3) a GROUP BY a.myCol PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-08_074_964366677410816892/-mr-10000 POSTHOOK: query: SELECT a.myCol, count(1) FROM (SELECT explode(array(1,2,3)) AS myCol FROM src LIMIT 3) a GROUP BY a.myCol POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-08_074_964366677410816892/-mr-10000 1 1 2 1 3 1 PREHOOK: query: EXPLAIN SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3 PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3 POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION explode (TOK_FUNCTION map 1 'one' 2 'two' 3 'three')) myKey myVal)) (TOK_LIMIT 3))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: src TableScan alias: src Select Operator expressions: expr: map(1:'one',2:'two',3:'three') type: map outputColumnNames: _col0 UDTF Operator function name: explode Limit File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat Stage: Stage-0 Fetch Operator limit: 3 PREHOOK: query: EXPLAIN EXTENDED SELECT a.myKey, a.myVal, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3) a GROUP BY a.myKey, a.myVal PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT a.myKey, a.myVal, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3) a GROUP BY a.myKey, a.myVal POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME src))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_FUNCTION explode (TOK_FUNCTION map 1 'one' 2 'two' 3 'three')) myKey myVal)) (TOK_LIMIT 3))) a)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) myKey)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) myVal)) (TOK_SELEXPR (TOK_FUNCTION count 1))) (TOK_GROUPBY (. (TOK_TABLE_OR_COL a) myKey) (. (TOK_TABLE_OR_COL a) myVal)))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-2 depends on stages: Stage-1 Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: a:src TableScan alias: src GatherStats: false Select Operator expressions: expr: map(1:'one',2:'two',3:'three') type: map outputColumnNames: _col0 UDTF Operator function name: explode Limit Reduce Output Operator sort order: tag: -1 value expressions: expr: key type: int expr: value type: string Needs Tagging: false Path -> Alias: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src [a:src] Path -> Partition: pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src Partition base file name: src input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types string:string file.inputformat org.apache.hadoop.mapred.TextInputFormat file.outputformat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat location pfile:/home/amarsri/workspace/hive/build/ql/test/data/warehouse/src name default.src serialization.ddl struct src { string key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe transient_lastDdlTime 1314078275 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.src name: default.src Reduce Operator Tree: Extract Limit Select Operator expressions: expr: _col0 type: int expr: _col1 type: string outputColumnNames: _col0, _col1 Group By Operator aggregations: expr: count(1) bucketGroup: false keys: expr: _col0 type: int expr: _col1 type: string mode: hash outputColumnNames: _col0, _col1, _col2 File Output Operator compressed: false GlobalTableId: 0 directory: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-mr-10002 NumFilesPerFileSink: 1 table: input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1,_col2 columns.types int,string,bigint escape.delim \ TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-2 Map Reduce Alias -> Map Operator Tree: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-mr-10002 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ Map-reduce partition columns: expr: _col0 type: int expr: _col1 type: string tag: -1 value expressions: expr: _col2 type: bigint Needs Tagging: false Path -> Alias: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-mr-10002 [file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-mr-10002] Path -> Partition: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-mr-10002 Partition base file name: -mr-10002 input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1,_col2 columns.types int,string,bigint escape.delim \ input format: org.apache.hadoop.mapred.SequenceFileInputFormat output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat properties: columns _col0,_col1,_col2 columns.types int,string,bigint escape.delim \ Reduce Operator Tree: Group By Operator aggregations: expr: count(VALUE._col0) bucketGroup: false keys: expr: KEY._col0 type: int expr: KEY._col1 type: string mode: mergepartial outputColumnNames: _col0, _col1, _col2 Select Operator expressions: expr: _col0 type: int expr: _col1 type: string expr: _col2 type: bigint outputColumnNames: _col0, _col1, _col2 File Output Operator compressed: false GlobalTableId: 0 directory: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-ext-10001 NumFilesPerFileSink: 1 Stats Publishing Key Prefix: file:/tmp/amarsri/hive_2011-08-23_03-07-13_126_71315112476316699/-ext-10001/ table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1,_col2 columns.types int:string:bigint escape.delim \ serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3 PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-13_174_7052821188998877991/-mr-10000 POSTHOOK: query: SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3 POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-13_174_7052821188998877991/-mr-10000 1 one 2 two 3 three PREHOOK: query: SELECT a.myKey, a.myVal, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3) a GROUP BY a.myKey, a.myVal PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-15_728_1285366516371227133/-mr-10000 POSTHOOK: query: SELECT a.myKey, a.myVal, count(1) FROM (SELECT explode(map(1,'one',2,'two',3,'three')) as (myKey,myVal) FROM src LIMIT 3) a GROUP BY a.myKey, a.myVal POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: file:/tmp/amarsri/hive_2011-08-23_03-07-15_728_1285366516371227133/-mr-10000 1 one 1 2 two 1 3 three 1