PREHOOK: query: drop table dest_j1 PREHOOK: type: DROPTABLE POSTHOOK: query: drop table dest_j1 POSTHOOK: type: DROPTABLE PREHOOK: query: drop table tmp1 PREHOOK: type: DROPTABLE POSTHOOK: query: drop table tmp1 POSTHOOK: type: DROPTABLE PREHOOK: query: drop table tmp2 PREHOOK: type: DROPTABLE POSTHOOK: query: drop table tmp2 POSTHOOK: type: DROPTABLE PREHOOK: query: CREATE TABLE tmp1(key INT, cnt INT) PREHOOK: type: CREATETABLE POSTHOOK: query: CREATE TABLE tmp1(key INT, cnt INT) POSTHOOK: type: CREATETABLE POSTHOOK: Output: default@tmp1 PREHOOK: query: CREATE TABLE tmp2(key INT, cnt INT) PREHOOK: type: CREATETABLE POSTHOOK: query: CREATE TABLE tmp2(key INT, cnt INT) POSTHOOK: type: CREATETABLE POSTHOOK: Output: default@tmp2 PREHOOK: query: CREATE TABLE dest_j1(key INT, value INT, val2 INT) PREHOOK: type: CREATETABLE POSTHOOK: query: CREATE TABLE dest_j1(key INT, value INT, val2 INT) POSTHOOK: type: CREATETABLE POSTHOOK: Output: default@dest_j1 PREHOOK: query: INSERT OVERWRITE TABLE tmp1 SELECT key, count(1) from src group by key PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@tmp1 POSTHOOK: query: INSERT OVERWRITE TABLE tmp1 SELECT key, count(1) from src group by key POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@tmp1 POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] PREHOOK: query: INSERT OVERWRITE TABLE tmp2 SELECT key, count(1) from src group by key PREHOOK: type: QUERY PREHOOK: Input: default@src PREHOOK: Output: default@tmp2 POSTHOOK: query: INSERT OVERWRITE TABLE tmp2 SELECT key, count(1) from src group by key POSTHOOK: type: QUERY POSTHOOK: Input: default@src POSTHOOK: Output: default@tmp2 POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: tmp2.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] PREHOOK: query: EXPLAIN INSERT OVERWRITE TABLE dest_j1 SELECT /*+ MAPJOIN(x) */ x.key, x.cnt, y.cnt FROM tmp1 x JOIN tmp2 y ON (x.key = y.key) PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN INSERT OVERWRITE TABLE dest_j1 SELECT /*+ MAPJOIN(x) */ x.key, x.cnt, y.cnt FROM tmp1 x JOIN tmp2 y ON (x.key = y.key) POSTHOOK: type: QUERY POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: tmp2.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF tmp1 x) (TOK_TABREF tmp2 y) (= (. (TOK_TABLE_OR_COL x) key) (. (TOK_TABLE_OR_COL y) key)))) (TOK_INSERT (TOK_DESTINATION (TOK_TAB dest_j1)) (TOK_SELECT (TOK_HINTLIST (TOK_HINT TOK_MAPJOIN (TOK_HINTARGLIST x))) (TOK_SELEXPR (. (TOK_TABLE_OR_COL x) key)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL x) cnt)) (TOK_SELEXPR (. (TOK_TABLE_OR_COL y) cnt))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-4 depends on stages: Stage-1 , consists of Stage-3, Stage-2 Stage-3 Stage-0 depends on stages: Stage-3, Stage-2 Stage-2 STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: y TableScan alias: y Common Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {key} {cnt} 1 {cnt} handleSkewJoin: false keys: 0 [Column[key]] 1 [Column[key]] outputColumnNames: _col0, _col1, _col3 Position of Big Table: 1 Select Operator expressions: expr: _col0 type: int expr: _col1 type: int expr: _col3 type: int outputColumnNames: _col0, _col1, _col3 Select Operator expressions: expr: _col0 type: int expr: _col1 type: int expr: _col3 type: int outputColumnNames: _col0, _col1, _col2 File Output Operator compressed: false GlobalTableId: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: dest_j1 Local Work: Map Reduce Local Work Alias -> Map Local Tables: x Fetch Operator limit: -1 Alias -> Map Local Operator Tree: x TableScan alias: x Common Join Operator condition map: Inner Join 0 to 1 condition expressions: 0 {key} {cnt} 1 {cnt} handleSkewJoin: false keys: 0 [Column[key]] 1 [Column[key]] outputColumnNames: _col0, _col1, _col3 Position of Big Table: 1 Select Operator expressions: expr: _col0 type: int expr: _col1 type: int expr: _col3 type: int outputColumnNames: _col0, _col1, _col3 Select Operator expressions: expr: _col0 type: int expr: _col1 type: int expr: _col3 type: int outputColumnNames: _col0, _col1, _col2 File Output Operator compressed: false GlobalTableId: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: dest_j1 Stage: Stage-4 Conditional Operator Stage: Stage-3 Move Operator files: hdfs directory: true destination: file:/data/users/nzhang/reviews/0.6/branch-0.6/build/ql/scratchdir/hive_2010-08-23_22-14-26_546_3279907980243548690/10000 Stage: Stage-0 Move Operator tables: replace: true table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: dest_j1 Stage: Stage-2 Map Reduce Alias -> Map Operator Tree: file:/data/users/nzhang/reviews/0.6/branch-0.6/build/ql/scratchdir/hive_2010-08-23_22-14-26_546_3279907980243548690/10002 Reduce Output Operator sort order: Map-reduce partition columns: expr: rand() type: double tag: -1 value expressions: expr: key type: int expr: value type: int expr: val2 type: int Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: dest_j1 PREHOOK: query: INSERT OVERWRITE TABLE dest_j1 SELECT /*+ MAPJOIN(x) */ x.key, x.cnt, y.cnt FROM tmp1 x JOIN tmp2 y ON (x.key = y.key) PREHOOK: type: QUERY PREHOOK: Input: default@tmp2 PREHOOK: Input: default@tmp1 PREHOOK: Output: default@dest_j1 POSTHOOK: query: INSERT OVERWRITE TABLE dest_j1 SELECT /*+ MAPJOIN(x) */ x.key, x.cnt, y.cnt FROM tmp1 x JOIN tmp2 y ON (x.key = y.key) POSTHOOK: type: QUERY POSTHOOK: Input: default@tmp2 POSTHOOK: Input: default@tmp1 POSTHOOK: Output: default@dest_j1 POSTHOOK: Lineage: dest_j1.key SIMPLE [(tmp1)x.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.val2 SIMPLE [(tmp2)y.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.value SIMPLE [(tmp1)x.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: tmp2.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] PREHOOK: query: select * from dest_j1 x order by x.key PREHOOK: type: QUERY PREHOOK: Input: default@dest_j1 PREHOOK: Output: file:/tmp/nzhang/hive_2010-08-23_22-14-29_994_8986891445716702677/10000 POSTHOOK: query: select * from dest_j1 x order by x.key POSTHOOK: type: QUERY POSTHOOK: Input: default@dest_j1 POSTHOOK: Output: file:/tmp/nzhang/hive_2010-08-23_22-14-29_994_8986891445716702677/10000 POSTHOOK: Lineage: dest_j1.key SIMPLE [(tmp1)x.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.val2 SIMPLE [(tmp2)y.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.value SIMPLE [(tmp1)x.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: tmp2.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] 0 3 3 2 1 1 4 1 1 5 3 3 8 1 1 9 1 1 10 1 1 11 1 1 12 2 2 15 2 2 17 1 1 18 2 2 19 1 1 20 1 1 24 2 2 26 2 2 27 1 1 28 1 1 30 1 1 33 1 1 34 1 1 35 3 3 37 2 2 41 1 1 42 2 2 43 1 1 44 1 1 47 1 1 51 2 2 53 1 1 54 1 1 57 1 1 58 2 2 64 1 1 65 1 1 66 1 1 67 2 2 69 1 1 70 3 3 72 2 2 74 1 1 76 2 2 77 1 1 78 1 1 80 1 1 82 1 1 83 2 2 84 2 2 85 1 1 86 1 1 87 1 1 90 3 3 92 1 1 95 2 2 96 1 1 97 2 2 98 2 2 100 2 2 103 2 2 104 2 2 105 1 1 111 1 1 113 2 2 114 1 1 116 1 1 118 2 2 119 3 3 120 2 2 125 2 2 126 1 1 128 3 3 129 2 2 131 1 1 133 1 1 134 2 2 136 1 1 137 2 2 138 4 4 143 1 1 145 1 1 146 2 2 149 2 2 150 1 1 152 2 2 153 1 1 155 1 1 156 1 1 157 1 1 158 1 1 160 1 1 162 1 1 163 1 1 164 2 2 165 2 2 166 1 1 167 3 3 168 1 1 169 4 4 170 1 1 172 2 2 174 2 2 175 2 2 176 2 2 177 1 1 178 1 1 179 2 2 180 1 1 181 1 1 183 1 1 186 1 1 187 3 3 189 1 1 190 1 1 191 2 2 192 1 1 193 3 3 194 1 1 195 2 2 196 1 1 197 2 2 199 3 3 200 2 2 201 1 1 202 1 1 203 2 2 205 2 2 207 2 2 208 3 3 209 2 2 213 2 2 214 1 1 216 2 2 217 2 2 218 1 1 219 2 2 221 2 2 222 1 1 223 2 2 224 2 2 226 1 1 228 1 1 229 2 2 230 5 5 233 2 2 235 1 1 237 2 2 238 2 2 239 2 2 241 1 1 242 2 2 244 1 1 247 1 1 248 1 1 249 1 1 252 1 1 255 2 2 256 2 2 257 1 1 258 1 1 260 1 1 262 1 1 263 1 1 265 2 2 266 1 1 272 2 2 273 3 3 274 1 1 275 1 1 277 4 4 278 2 2 280 2 2 281 2 2 282 2 2 283 1 1 284 1 1 285 1 1 286 1 1 287 1 1 288 2 2 289 1 1 291 1 1 292 1 1 296 1 1 298 3 3 302 1 1 305 1 1 306 1 1 307 2 2 308 1 1 309 2 2 310 1 1 311 3 3 315 1 1 316 3 3 317 2 2 318 3 3 321 2 2 322 2 2 323 1 1 325 2 2 327 3 3 331 2 2 332 1 1 333 2 2 335 1 1 336 1 1 338 1 1 339 1 1 341 1 1 342 2 2 344 2 2 345 1 1 348 5 5 351 1 1 353 2 2 356 1 1 360 1 1 362 1 1 364 1 1 365 1 1 366 1 1 367 2 2 368 1 1 369 3 3 373 1 1 374 1 1 375 1 1 377 1 1 378 1 1 379 1 1 382 2 2 384 3 3 386 1 1 389 1 1 392 1 1 393 1 1 394 1 1 395 2 2 396 3 3 397 2 2 399 2 2 400 1 1 401 5 5 402 1 1 403 3 3 404 2 2 406 4 4 407 1 1 409 3 3 411 1 1 413 2 2 414 2 2 417 3 3 418 1 1 419 1 1 421 1 1 424 2 2 427 1 1 429 2 2 430 3 3 431 3 3 432 1 1 435 1 1 436 1 1 437 1 1 438 3 3 439 2 2 443 1 1 444 1 1 446 1 1 448 1 1 449 1 1 452 1 1 453 1 1 454 3 3 455 1 1 457 1 1 458 2 2 459 2 2 460 1 1 462 2 2 463 2 2 466 3 3 467 1 1 468 4 4 469 5 5 470 1 1 472 1 1 475 1 1 477 1 1 478 2 2 479 1 1 480 3 3 481 1 1 482 1 1 483 1 1 484 1 1 485 1 1 487 1 1 489 4 4 490 1 1 491 1 1 492 2 2 493 1 1 494 1 1 495 1 1 496 1 1 497 1 1 498 3 3 PREHOOK: query: drop table dest_j1 PREHOOK: type: DROPTABLE POSTHOOK: query: drop table dest_j1 POSTHOOK: type: DROPTABLE POSTHOOK: Output: default@dest_j1 POSTHOOK: Lineage: dest_j1.key SIMPLE [(tmp1)x.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.val2 SIMPLE [(tmp2)y.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: dest_j1.value SIMPLE [(tmp1)x.FieldSchema(name:cnt, type:int, comment:null), ] POSTHOOK: Lineage: tmp1.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp1.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ] POSTHOOK: Lineage: tmp2.cnt EXPRESSION [(src)src.null, ] POSTHOOK: Lineage: tmp2.key EXPRESSION [(src)src.FieldSchema(name:key, type:string, comment:default), ]