PREHOOK: query: CREATE TABLE dest1(key INT, value STRING) STORED AS TEXTFILE PREHOOK: type: CREATETABLE POSTHOOK: query: CREATE TABLE dest1(key INT, value STRING) STORED AS TEXTFILE POSTHOOK: type: CREATETABLE POSTHOOK: Output: default@dest1 PREHOOK: query: -- both input pruning and sample filter EXPLAIN EXTENDED INSERT OVERWRITE TABLE dest1 SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 4 on key) s PREHOOK: type: QUERY POSTHOOK: query: -- both input pruning and sample filter EXPLAIN EXTENDED INSERT OVERWRITE TABLE dest1 SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 4 on key) s POSTHOOK: type: QUERY ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket) (TOK_TABLEBUCKETSAMPLE 1 4 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_TAB (TOK_TABNAME dest1))) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-7 depends on stages: Stage-1 , consists of Stage-4, Stage-3, Stage-5 Stage-4 Stage-0 depends on stages: Stage-4, Stage-3, Stage-6 Stage-2 depends on stages: Stage-0 Stage-3 Stage-5 Stage-6 depends on stages: Stage-5 STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 4) = 0) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 File Output Operator compressed: false GlobalTableId: 1 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 TotalFiles: 1 GatherStats: true MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket0.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket name: default.srcbucket Truncated Path -> Alias: /srcbucket/srcbucket0.txt [s] Stage: Stage-7 Conditional Operator Stage: Stage-4 Move Operator files: hdfs directory: true #### A masked pattern was here #### Stage: Stage-0 Move Operator tables: replace: true #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 #### A masked pattern was here #### Stage: Stage-2 Stats-Aggr Operator #### A masked pattern was here #### Stage: Stage-3 Map Reduce Alias -> Map Operator Tree: #### A masked pattern was here #### File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: -ext-10002 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 name: default.dest1 Truncated Path -> Alias: #### A masked pattern was here #### Stage: Stage-5 Map Reduce Alias -> Map Operator Tree: #### A masked pattern was here #### File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: -ext-10002 input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count -1 columns key,value columns.types int:string #### A masked pattern was here #### name default.dest1 serialization.ddl struct dest1 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.dest1 name: default.dest1 Truncated Path -> Alias: #### A masked pattern was here #### Stage: Stage-6 Move Operator files: hdfs directory: true #### A masked pattern was here #### PREHOOK: query: INSERT OVERWRITE TABLE dest1 SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 4 on key) s PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket PREHOOK: Output: default@dest1 POSTHOOK: query: INSERT OVERWRITE TABLE dest1 SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 4 on key) s POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket POSTHOOK: Output: default@dest1 POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] PREHOOK: query: SELECT dest1.* FROM dest1 PREHOOK: type: QUERY PREHOOK: Input: default@dest1 #### A masked pattern was here #### POSTHOOK: query: SELECT dest1.* FROM dest1 POSTHOOK: type: QUERY POSTHOOK: Input: default@dest1 #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 468 val_469 272 val_273 448 val_449 440 val_441 296 val_297 428 val_429 356 val_357 128 val_129 0 val_1 240 val_241 408 val_409 476 val_477 48 val_49 424 val_425 488 val_489 128 val_129 468 val_469 224 val_225 344 val_345 4 val_5 56 val_57 304 val_305 264 val_265 196 val_197 20 val_21 492 val_493 360 val_361 68 val_69 16 val_17 492 val_493 376 val_377 120 val_121 132 val_133 388 val_389 184 val_185 284 val_285 352 val_353 328 val_329 480 val_481 480 val_481 392 val_393 476 val_477 252 val_253 264 val_265 48 val_49 336 val_337 340 val_341 484 val_485 260 val_261 164 val_165 104 val_105 80 val_81 140 val_141 212 val_213 308 val_309 416 val_417 364 val_365 20 val_21 52 val_53 40 val_41 8 val_9 168 val_169 384 val_385 324 val_325 404 val_405 260 val_261 328 val_329 404 val_405 384 val_385 76 val_77 116 val_117 104 val_105 32 val_33 132 val_133 192 val_193 356 val_357 352 val_353 52 val_53 160 val_161 76 val_77 412 val_413 16 val_17 204 val_205 216 val_217 196 val_197 12 val_13 384 val_385 60 val_61 52 val_53 404 val_405 300 val_301 0 val_1 268 val_269 392 val_393 104 val_105 436 val_437 156 val_157 172 val_173 244 val_245 284 val_285 164 val_165 136 val_137 432 val_433 496 val_497 144 val_145 408 val_409 152 val_153 348 val_349 292 val_293 52 val_53 152 val_153 256 val_257 292 val_293 412 val_413 40 val_41 100 val_101 156 val_157 228 val_229 248 val_249 244 val_245 276 val_277 196 val_197 440 val_441 100 val_101 308 val_309 468 val_469 152 val_153 76 val_77 300 val_301 244 val_245 484 val_484 224 val_224 128 val_128 152 val_152 252 val_252 292 val_292 208 val_208 396 val_396 0 val_0 128 val_128 316 val_316 20 val_20 92 val_92 72 val_72 4 val_4 280 val_280 208 val_208 356 val_356 192 val_192 176 val_176 216 val_216 176 val_176 332 val_332 180 val_180 284 val_284 12 val_12 260 val_260 404 val_404 384 val_384 272 val_272 84 val_84 348 val_348 8 val_8 208 val_208 348 val_348 24 val_24 172 val_172 496 val_496 0 val_0 468 val_468 100 val_100 96 val_96 120 val_120 404 val_404 436 val_436 156 val_156 468 val_468 308 val_308 196 val_196 288 val_288 316 val_316 0 val_0 364 val_364 72 val_72 224 val_224 392 val_392 272 val_272 452 val_452 396 val_396 336 val_336 168 val_168 472 val_472 160 val_160 76 val_76 492 val_492 228 val_228 64 val_64 468 val_468 76 val_76 368 val_368 296 val_296 216 val_216 344 val_344 116 val_116 256 val_256 480 val_480 288 val_288 244 val_244 128 val_128 432 val_432 316 val_316 280 val_280 80 val_80 44 val_44 104 val_104 348 val_348 424 val_424 12 val_12 396 val_396 164 val_164 164 val_164 424 val_424 480 val_480 24 val_24 104 val_104 200 val_200 360 val_360 248 val_248 444 val_444 120 val_120 468 val_468 460 val_460 480 val_480 136 val_136 172 val_172 384 val_384 256 val_256 384 val_384 492 val_492 100 val_100 348 val_348 344 val_344 84 val_84 28 val_28 448 val_448 152 val_152 348 val_348 400 val_400 200 val_200 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 4 OUT OF 4 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 4 OUT OF 4 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket) (TOK_TABLEBUCKETSAMPLE 4 4 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 4) = 3) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket1.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket name: default.srcbucket Truncated Path -> Alias: /srcbucket/srcbucket1.txt [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 4 OUT OF 4 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 4 OUT OF 4 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 3 val_4 11 val_11 11 val_12 11 val_12 11 val_12 15 val_15 15 val_15 15 val_16 15 val_16 19 val_19 19 val_20 23 val_24 27 val_27 31 val_32 35 val_35 35 val_35 35 val_35 35 val_36 35 val_36 35 val_36 43 val_43 47 val_47 47 val_48 51 val_51 51 val_51 51 val_52 59 val_60 63 val_64 67 val_67 67 val_67 71 val_72 75 val_76 83 val_83 83 val_83 87 val_87 87 val_88 87 val_88 91 val_92 95 val_95 95 val_95 99 val_100 103 val_103 103 val_103 111 val_111 119 val_119 119 val_119 119 val_119 119 val_120 119 val_120 119 val_120 123 val_124 123 val_124 131 val_131 135 val_136 135 val_136 135 val_136 143 val_143 143 val_144 147 val_148 147 val_148 151 val_152 151 val_152 155 val_155 163 val_163 167 val_167 167 val_167 167 val_167 167 val_168 175 val_175 175 val_175 175 val_176 175 val_176 179 val_179 179 val_179 179 val_180 183 val_183 183 val_184 187 val_187 187 val_187 187 val_187 191 val_191 191 val_191 191 val_192 195 val_195 195 val_195 199 val_199 199 val_199 199 val_199 199 val_200 203 val_203 203 val_203 207 val_207 207 val_207 215 val_216 219 val_219 219 val_219 223 val_223 223 val_223 227 val_228 231 val_232 235 val_235 235 val_236 239 val_239 239 val_239 239 val_240 239 val_240 243 val_244 243 val_244 247 val_247 255 val_255 255 val_255 259 val_260 259 val_260 263 val_263 267 val_268 271 val_272 275 val_275 275 val_276 275 val_276 283 val_283 287 val_287 287 val_288 287 val_288 291 val_291 291 val_292 291 val_292 295 val_296 295 val_296 303 val_304 303 val_304 307 val_307 307 val_307 311 val_311 311 val_311 311 val_311 315 val_315 323 val_323 323 val_324 327 val_327 327 val_327 327 val_327 331 val_331 331 val_331 331 val_332 331 val_332 335 val_335 335 val_336 335 val_336 339 val_339 343 val_344 347 val_348 347 val_348 351 val_351 351 val_352 351 val_352 355 val_356 355 val_356 363 val_364 363 val_364 363 val_364 367 val_367 367 val_367 367 val_368 367 val_368 371 val_372 371 val_372 371 val_372 371 val_372 375 val_375 375 val_376 375 val_376 375 val_376 375 val_376 375 val_376 379 val_379 379 val_380 391 val_392 391 val_392 395 val_395 395 val_395 395 val_396 395 val_396 399 val_399 399 val_399 399 val_400 399 val_400 403 val_403 403 val_403 403 val_403 407 val_407 407 val_408 407 val_408 407 val_408 411 val_411 411 val_412 415 val_416 419 val_419 423 val_424 427 val_427 427 val_428 427 val_428 431 val_431 431 val_431 431 val_431 431 val_432 435 val_435 435 val_436 439 val_439 439 val_439 439 val_440 439 val_440 443 val_443 443 val_444 443 val_444 443 val_444 447 val_448 451 val_452 455 val_455 455 val_456 455 val_456 459 val_459 459 val_459 459 val_460 463 val_463 463 val_463 463 val_464 467 val_467 467 val_468 475 val_475 475 val_476 479 val_479 483 val_483 487 val_487 487 val_488 491 val_491 491 val_492 491 val_492 495 val_495 495 val_496 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket) (TOK_TABLEBUCKETSAMPLE 1 2 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 2) = 0) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket0.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket name: default.srcbucket Truncated Path -> Alias: /srcbucket/srcbucket0.txt [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 0 val_0 0 val_0 0 val_0 0 val_1 0 val_1 2 val_2 2 val_3 4 val_4 4 val_5 6 val_7 6 val_7 8 val_8 8 val_9 10 val_10 10 val_11 12 val_12 12 val_12 12 val_13 16 val_17 16 val_17 18 val_18 18 val_18 20 val_20 20 val_21 20 val_21 22 val_23 24 val_24 24 val_24 26 val_26 26 val_26 28 val_28 30 val_30 30 val_31 32 val_33 34 val_34 40 val_41 40 val_41 42 val_42 42 val_42 42 val_43 42 val_43 42 val_43 44 val_44 46 val_47 48 val_49 48 val_49 50 val_51 52 val_53 52 val_53 52 val_53 52 val_53 54 val_54 56 val_57 58 val_58 58 val_58 58 val_59 58 val_59 60 val_61 62 val_63 62 val_63 64 val_64 66 val_66 68 val_69 70 val_70 70 val_70 70 val_70 70 val_71 72 val_72 72 val_72 74 val_74 76 val_76 76 val_76 76 val_77 76 val_77 76 val_77 78 val_78 78 val_79 80 val_80 80 val_81 82 val_82 82 val_83 82 val_83 84 val_84 84 val_84 86 val_86 86 val_87 90 val_90 90 val_90 90 val_90 92 val_92 94 val_95 96 val_96 98 val_98 98 val_98 100 val_100 100 val_100 100 val_101 100 val_101 102 val_103 104 val_104 104 val_104 104 val_105 104 val_105 104 val_105 106 val_107 110 val_111 114 val_114 114 val_115 114 val_115 114 val_115 116 val_116 116 val_117 118 val_118 118 val_118 118 val_119 118 val_119 118 val_119 120 val_120 120 val_120 120 val_121 122 val_123 122 val_123 122 val_123 126 val_126 126 val_127 126 val_127 128 val_128 128 val_128 128 val_128 128 val_129 128 val_129 132 val_133 132 val_133 134 val_134 134 val_134 134 val_135 136 val_136 136 val_137 138 val_138 138 val_138 138 val_138 138 val_138 138 val_139 138 val_139 140 val_141 144 val_145 146 val_146 146 val_146 150 val_150 152 val_152 152 val_152 152 val_153 152 val_153 152 val_153 156 val_156 156 val_157 156 val_157 158 val_158 160 val_160 160 val_161 162 val_162 162 val_163 164 val_164 164 val_164 164 val_165 164 val_165 166 val_166 168 val_168 168 val_169 170 val_170 170 val_171 172 val_172 172 val_172 172 val_173 174 val_174 174 val_174 174 val_175 174 val_175 176 val_176 176 val_176 178 val_178 178 val_179 178 val_179 180 val_180 182 val_183 184 val_185 186 val_186 190 val_190 192 val_192 192 val_193 194 val_194 196 val_196 196 val_197 196 val_197 196 val_197 200 val_200 200 val_200 202 val_202 204 val_205 206 val_207 206 val_207 206 val_207 208 val_208 208 val_208 208 val_208 212 val_213 214 val_214 216 val_216 216 val_216 216 val_217 218 val_218 222 val_222 222 val_223 224 val_224 224 val_224 224 val_225 226 val_226 226 val_227 226 val_227 226 val_227 226 val_227 228 val_228 228 val_229 230 val_230 230 val_230 230 val_230 230 val_230 230 val_230 238 val_238 238 val_238 238 val_239 240 val_241 242 val_242 242 val_242 242 val_243 244 val_244 244 val_245 244 val_245 244 val_245 246 val_247 246 val_247 248 val_248 248 val_249 252 val_252 252 val_253 254 val_255 256 val_256 256 val_256 256 val_257 258 val_258 258 val_259 260 val_260 260 val_261 260 val_261 262 val_262 262 val_263 262 val_263 264 val_265 264 val_265 266 val_266 268 val_269 272 val_272 272 val_272 272 val_273 274 val_274 274 val_275 276 val_277 278 val_278 278 val_278 278 val_279 280 val_280 280 val_280 282 val_282 282 val_282 284 val_284 284 val_285 284 val_285 286 val_286 286 val_287 288 val_288 288 val_288 292 val_292 292 val_293 292 val_293 296 val_296 296 val_297 298 val_298 298 val_298 298 val_298 300 val_301 300 val_301 302 val_302 302 val_303 304 val_305 306 val_306 306 val_307 308 val_308 308 val_309 308 val_309 310 val_310 310 val_311 310 val_311 310 val_311 314 val_315 316 val_316 316 val_316 316 val_316 318 val_318 318 val_318 318 val_318 318 val_319 322 val_322 322 val_322 322 val_323 324 val_325 326 val_327 328 val_329 328 val_329 330 val_331 332 val_332 334 val_335 336 val_336 336 val_337 338 val_338 338 val_339 340 val_341 342 val_342 342 val_342 342 val_343 344 val_344 344 val_344 344 val_345 348 val_348 348 val_348 348 val_348 348 val_348 348 val_348 348 val_349 352 val_353 352 val_353 356 val_356 356 val_357 356 val_357 358 val_359 360 val_360 360 val_361 362 val_362 364 val_364 364 val_365 366 val_366 368 val_368 374 val_374 374 val_375 376 val_377 378 val_378 378 val_379 382 val_382 382 val_382 382 val_383 382 val_383 384 val_384 384 val_384 384 val_384 384 val_385 384 val_385 384 val_385 386 val_386 386 val_387 386 val_387 388 val_389 390 val_391 390 val_391 390 val_391 392 val_392 392 val_393 392 val_393 394 val_394 396 val_396 396 val_396 396 val_396 398 val_399 400 val_400 402 val_402 402 val_403 402 val_403 402 val_403 404 val_404 404 val_404 404 val_405 404 val_405 404 val_405 406 val_406 406 val_406 406 val_406 406 val_406 406 val_407 408 val_409 408 val_409 410 val_411 412 val_413 412 val_413 414 val_414 414 val_414 414 val_415 416 val_417 418 val_418 424 val_424 424 val_424 424 val_425 426 val_427 428 val_429 430 val_430 430 val_430 430 val_430 430 val_431 432 val_432 432 val_433 436 val_436 436 val_437 438 val_438 438 val_438 438 val_438 438 val_439 438 val_439 440 val_441 440 val_441 442 val_443 444 val_444 446 val_446 446 val_447 446 val_447 448 val_448 448 val_449 450 val_451 450 val_451 452 val_452 454 val_454 454 val_454 454 val_454 454 val_455 454 val_455 458 val_458 458 val_458 460 val_460 462 val_462 462 val_462 462 val_463 466 val_466 466 val_466 466 val_466 468 val_468 468 val_468 468 val_468 468 val_468 468 val_469 468 val_469 468 val_469 470 val_470 470 val_471 472 val_472 474 val_475 474 val_475 476 val_477 476 val_477 478 val_478 478 val_478 478 val_479 478 val_479 480 val_480 480 val_480 480 val_480 480 val_481 480 val_481 482 val_482 482 val_483 484 val_484 484 val_485 486 val_487 488 val_489 490 val_490 490 val_491 492 val_492 492 val_492 492 val_493 492 val_493 494 val_494 494 val_495 494 val_495 496 val_496 496 val_497 498 val_498 498 val_498 498 val_498 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 3 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 3 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket) (TOK_TABLEBUCKETSAMPLE 1 3 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 3) = 0) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket name: default.srcbucket Truncated Path -> Alias: /srcbucket [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 3 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 1 OUT OF 3 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 0 val_0 0 val_0 0 val_0 0 val_1 0 val_1 3 val_4 6 val_7 6 val_7 9 val_9 12 val_12 12 val_12 12 val_13 15 val_15 15 val_15 15 val_16 15 val_16 18 val_18 18 val_18 21 val_22 21 val_22 21 val_22 21 val_22 24 val_24 24 val_24 27 val_27 30 val_30 30 val_31 33 val_33 33 val_34 42 val_42 42 val_42 42 val_43 42 val_43 42 val_43 48 val_49 48 val_49 51 val_51 51 val_51 51 val_52 54 val_54 57 val_57 60 val_61 63 val_64 66 val_66 69 val_69 69 val_70 72 val_72 72 val_72 75 val_76 78 val_78 78 val_79 84 val_84 84 val_84 87 val_87 87 val_88 87 val_88 90 val_90 90 val_90 90 val_90 93 val_94 93 val_94 93 val_94 96 val_96 99 val_100 102 val_103 105 val_105 105 val_106 105 val_106 111 val_111 114 val_114 114 val_115 114 val_115 114 val_115 117 val_118 117 val_118 120 val_120 120 val_120 120 val_121 123 val_124 123 val_124 126 val_126 126 val_127 126 val_127 129 val_129 129 val_129 129 val_130 129 val_130 132 val_133 132 val_133 135 val_136 135 val_136 135 val_136 138 val_138 138 val_138 138 val_138 138 val_138 138 val_139 138 val_139 144 val_145 147 val_148 147 val_148 150 val_150 153 val_153 153 val_154 153 val_154 156 val_156 156 val_157 156 val_157 162 val_162 162 val_163 165 val_165 165 val_165 165 val_166 168 val_168 168 val_169 174 val_174 174 val_174 174 val_175 174 val_175 177 val_177 177 val_178 177 val_178 180 val_180 183 val_183 183 val_184 186 val_186 189 val_189 189 val_190 192 val_192 192 val_193 195 val_195 195 val_195 201 val_201 204 val_205 207 val_207 207 val_207 213 val_213 213 val_213 213 val_214 216 val_216 216 val_216 216 val_217 219 val_219 219 val_219 222 val_222 222 val_223 228 val_228 228 val_229 231 val_232 237 val_237 237 val_237 240 val_241 243 val_244 243 val_244 246 val_247 246 val_247 249 val_249 249 val_250 249 val_250 252 val_252 252 val_253 255 val_255 255 val_255 258 val_258 258 val_259 261 val_262 264 val_265 264 val_265 267 val_268 273 val_273 273 val_273 273 val_273 273 val_274 276 val_277 282 val_282 282 val_282 285 val_285 285 val_286 288 val_288 288 val_288 291 val_291 291 val_292 291 val_292 300 val_301 300 val_301 303 val_304 303 val_304 306 val_306 306 val_307 309 val_309 309 val_309 309 val_310 315 val_315 318 val_318 318 val_318 318 val_318 318 val_319 321 val_321 321 val_321 324 val_325 327 val_327 327 val_327 327 val_327 330 val_331 333 val_333 333 val_333 333 val_334 336 val_336 336 val_337 339 val_339 342 val_342 342 val_342 342 val_343 345 val_345 348 val_348 348 val_348 348 val_348 348 val_348 348 val_348 348 val_349 351 val_351 351 val_352 351 val_352 360 val_360 360 val_361 363 val_364 363 val_364 363 val_364 366 val_366 369 val_369 369 val_369 369 val_369 369 val_370 375 val_375 375 val_376 375 val_376 375 val_376 375 val_376 375 val_376 378 val_378 378 val_379 381 val_382 384 val_384 384 val_384 384 val_384 384 val_385 384 val_385 384 val_385 390 val_391 390 val_391 390 val_391 393 val_393 393 val_394 393 val_394 396 val_396 396 val_396 396 val_396 399 val_399 399 val_399 399 val_400 399 val_400 402 val_402 402 val_403 402 val_403 402 val_403 405 val_406 408 val_409 408 val_409 411 val_411 411 val_412 414 val_414 414 val_414 414 val_415 417 val_417 417 val_417 417 val_417 423 val_424 426 val_427 429 val_429 429 val_429 429 val_430 429 val_430 432 val_432 432 val_433 435 val_435 435 val_436 438 val_438 438 val_438 438 val_438 438 val_439 438 val_439 441 val_442 444 val_444 447 val_448 450 val_451 450 val_451 453 val_453 453 val_454 459 val_459 459 val_459 459 val_460 462 val_462 462 val_462 462 val_463 468 val_468 468 val_468 468 val_468 468 val_468 468 val_469 468 val_469 468 val_469 474 val_475 474 val_475 477 val_477 477 val_478 480 val_480 480 val_480 480 val_480 480 val_481 480 val_481 483 val_483 486 val_487 489 val_489 489 val_489 489 val_489 489 val_489 489 val_490 492 val_492 492 val_492 492 val_493 492 val_493 495 val_495 495 val_496 498 val_498 498 val_498 498 val_498 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 2 OUT OF 3 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 2 OUT OF 3 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket) (TOK_TABLEBUCKETSAMPLE 2 3 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 3) = 1) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 2 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket numFiles 2 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 11603 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket name: default.srcbucket Truncated Path -> Alias: /srcbucket [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 2 OUT OF 3 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket TABLESAMPLE (BUCKET 2 OUT OF 3 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 1 val_2 4 val_4 4 val_5 10 val_10 10 val_11 16 val_17 16 val_17 19 val_19 19 val_20 22 val_23 28 val_28 31 val_32 34 val_34 37 val_37 37 val_37 40 val_41 40 val_41 43 val_43 46 val_47 49 val_50 49 val_50 52 val_53 52 val_53 52 val_53 52 val_53 58 val_58 58 val_58 58 val_59 58 val_59 61 val_62 64 val_64 67 val_67 67 val_67 70 val_70 70 val_70 70 val_70 70 val_71 76 val_76 76 val_76 76 val_77 76 val_77 76 val_77 82 val_82 82 val_83 82 val_83 85 val_85 85 val_86 91 val_92 94 val_95 97 val_97 97 val_97 97 val_98 97 val_98 100 val_100 100 val_100 100 val_101 100 val_101 103 val_103 103 val_103 106 val_107 118 val_118 118 val_118 118 val_119 118 val_119 118 val_119 121 val_122 121 val_122 133 val_133 133 val_134 136 val_136 136 val_137 145 val_145 151 val_152 151 val_152 157 val_157 157 val_158 157 val_158 160 val_160 160 val_161 163 val_163 166 val_166 169 val_169 169 val_169 169 val_169 169 val_169 172 val_172 172 val_172 172 val_173 175 val_175 175 val_175 175 val_176 175 val_176 178 val_178 178 val_179 178 val_179 181 val_181 184 val_185 187 val_187 187 val_187 187 val_187 190 val_190 193 val_193 193 val_193 193 val_193 196 val_196 196 val_197 196 val_197 196 val_197 199 val_199 199 val_199 199 val_199 199 val_200 202 val_202 205 val_205 205 val_205 205 val_206 208 val_208 208 val_208 208 val_208 214 val_214 217 val_217 217 val_217 217 val_218 217 val_218 223 val_223 223 val_223 226 val_226 226 val_227 226 val_227 226 val_227 226 val_227 229 val_229 229 val_229 235 val_235 235 val_236 238 val_238 238 val_238 238 val_239 241 val_241 241 val_242 241 val_242 241 val_242 241 val_242 244 val_244 244 val_245 244 val_245 244 val_245 247 val_247 256 val_256 256 val_256 256 val_257 259 val_260 259 val_260 262 val_262 262 val_263 262 val_263 265 val_265 265 val_265 265 val_266 268 val_269 271 val_272 274 val_274 274 val_275 277 val_277 277 val_277 277 val_277 277 val_277 277 val_278 277 val_278 280 val_280 280 val_280 283 val_283 286 val_286 286 val_287 289 val_289 289 val_290 292 val_292 292 val_293 292 val_293 295 val_296 295 val_296 298 val_298 298 val_298 298 val_298 304 val_305 307 val_307 307 val_307 310 val_310 310 val_311 310 val_311 310 val_311 313 val_314 316 val_316 316 val_316 316 val_316 322 val_322 322 val_322 322 val_323 325 val_325 325 val_325 328 val_329 328 val_329 331 val_331 331 val_331 331 val_332 331 val_332 334 val_335 337 val_338 340 val_341 343 val_344 349 val_350 349 val_350 349 val_350 349 val_350 352 val_353 352 val_353 355 val_356 355 val_356 358 val_359 364 val_364 364 val_365 367 val_367 367 val_367 367 val_368 367 val_368 373 val_373 373 val_374 376 val_377 379 val_379 379 val_380 382 val_382 382 val_382 382 val_383 382 val_383 385 val_386 385 val_386 388 val_389 391 val_392 391 val_392 394 val_394 397 val_397 397 val_397 400 val_400 403 val_403 403 val_403 403 val_403 406 val_406 406 val_406 406 val_406 406 val_406 406 val_407 409 val_409 409 val_409 409 val_409 409 val_410 409 val_410 412 val_413 412 val_413 415 val_416 418 val_418 421 val_421 421 val_422 421 val_422 424 val_424 424 val_424 424 val_425 427 val_427 427 val_428 427 val_428 430 val_430 430 val_430 430 val_430 430 val_431 436 val_436 436 val_437 439 val_439 439 val_439 439 val_440 439 val_440 442 val_443 448 val_448 448 val_449 451 val_452 454 val_454 454 val_454 454 val_454 454 val_455 454 val_455 457 val_457 457 val_458 457 val_458 460 val_460 463 val_463 463 val_463 463 val_464 466 val_466 466 val_466 466 val_466 469 val_469 469 val_469 469 val_469 469 val_469 469 val_469 469 val_470 472 val_472 475 val_475 475 val_476 478 val_478 478 val_478 478 val_479 478 val_479 481 val_481 481 val_482 484 val_484 484 val_485 487 val_487 487 val_488 490 val_490 490 val_491 493 val_493 496 val_496 496 val_497 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket2) (TOK_TABLEBUCKETSAMPLE 1 2 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 2) = 0) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket20.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket2 name: default.srcbucket2 #### A masked pattern was here #### Partition base file name: srcbucket22.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket2 name: default.srcbucket2 Truncated Path -> Alias: /srcbucket2/srcbucket20.txt [s] /srcbucket2/srcbucket22.txt [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket2 #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket2 #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 0 val_0 0 val_0 0 val_0 2 val_2 4 val_4 8 val_8 20 val_20 24 val_24 24 val_24 26 val_26 26 val_26 28 val_28 42 val_42 42 val_42 44 val_44 64 val_64 66 val_66 80 val_80 82 val_82 84 val_84 84 val_84 86 val_86 114 val_114 116 val_116 118 val_118 118 val_118 134 val_134 134 val_134 136 val_136 138 val_138 138 val_138 138 val_138 138 val_138 150 val_150 152 val_152 152 val_152 156 val_156 158 val_158 170 val_170 172 val_172 172 val_172 174 val_174 174 val_174 176 val_176 176 val_176 178 val_178 190 val_190 192 val_192 194 val_194 196 val_196 200 val_200 200 val_200 202 val_202 208 val_208 208 val_208 208 val_208 222 val_222 224 val_224 224 val_224 226 val_226 228 val_228 242 val_242 242 val_242 244 val_244 248 val_248 260 val_260 262 val_262 266 val_266 280 val_280 280 val_280 282 val_282 282 val_282 284 val_284 286 val_286 288 val_288 288 val_288 310 val_310 316 val_316 316 val_316 316 val_316 318 val_318 318 val_318 318 val_318 332 val_332 336 val_336 338 val_338 356 val_356 374 val_374 378 val_378 392 val_392 394 val_394 396 val_396 396 val_396 396 val_396 400 val_400 402 val_402 404 val_404 404 val_404 406 val_406 406 val_406 406 val_406 406 val_406 424 val_424 424 val_424 444 val_444 446 val_446 448 val_448 460 val_460 462 val_462 462 val_462 466 val_466 466 val_466 466 val_466 468 val_468 468 val_468 468 val_468 468 val_468 480 val_480 480 val_480 480 val_480 482 val_482 484 val_484 PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 2 OUT OF 4 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 2 OUT OF 4 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME srcbucket2) (TOK_TABLEBUCKETSAMPLE 2 4 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 4) = 1) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Path -> Alias: #### A masked pattern was here #### Path -> Partition: #### A masked pattern was here #### Partition base file name: srcbucket21.txt input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: bucket_count 4 bucket_field_name key columns key,value columns.types int:string #### A masked pattern was here #### name default.srcbucket2 numFiles 4 numPartitions 0 numRows 0 rawDataSize 0 serialization.ddl struct srcbucket2 { i32 key, string value} serialization.format 1 serialization.lib org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe totalSize 5812 #### A masked pattern was here #### serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe name: default.srcbucket2 name: default.srcbucket2 Truncated Path -> Alias: /srcbucket2/srcbucket21.txt [s] Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 2 OUT OF 4 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@srcbucket2 #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM srcbucket2 TABLESAMPLE (BUCKET 2 OUT OF 4 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@srcbucket2 #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] 5 val_5 5 val_5 5 val_5 9 val_9 41 val_41 85 val_85 133 val_133 137 val_137 137 val_137 177 val_177 221 val_221 221 val_221 229 val_229 229 val_229 265 val_265 265 val_265 317 val_317 317 val_317 353 val_353 353 val_353 393 val_393 397 val_397 397 val_397 401 val_401 401 val_401 401 val_401 401 val_401 401 val_401 409 val_409 409 val_409 409 val_409 449 val_449 481 val_481 485 val_485 489 val_489 489 val_489 489 val_489 489 val_489 PREHOOK: query: CREATE TABLE empty_bucket (key int, value string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE PREHOOK: type: CREATETABLE POSTHOOK: query: CREATE TABLE empty_bucket (key int, value string) CLUSTERED BY (key) INTO 2 BUCKETS STORED AS TEXTFILE POSTHOOK: type: CREATETABLE POSTHOOK: Output: default@empty_bucket POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] PREHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM empty_bucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY POSTHOOK: query: EXPLAIN EXTENDED SELECT s.* FROM empty_bucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ] ABSTRACT SYNTAX TREE: (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME empty_bucket) (TOK_TABLEBUCKETSAMPLE 1 2 (TOK_TABLE_OR_COL key)) s)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME s)))) (TOK_ORDERBY (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL key)) (TOK_TABSORTCOLNAMEASC (TOK_TABLE_OR_COL value))))) STAGE DEPENDENCIES: Stage-1 is a root stage Stage-0 is a root stage STAGE PLANS: Stage: Stage-1 Map Reduce Alias -> Map Operator Tree: s TableScan alias: s GatherStats: false Filter Operator isSamplingPred: true predicate: expr: (((hash(key) & 2147483647) % 2) = 0) type: boolean Select Operator expressions: expr: key type: int expr: value type: string outputColumnNames: _col0, _col1 Reduce Output Operator key expressions: expr: _col0 type: int expr: _col1 type: string sort order: ++ tag: -1 value expressions: expr: _col0 type: int expr: _col1 type: string Needs Tagging: false Reduce Operator Tree: Extract File Output Operator compressed: false GlobalTableId: 0 #### A masked pattern was here #### NumFilesPerFileSink: 1 #### A masked pattern was here #### table: input format: org.apache.hadoop.mapred.TextInputFormat output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat properties: columns _col0,_col1 columns.types int:string escape.delim \ hive.serialization.extend.nesting.levels true serialization.format 1 TotalFiles: 1 GatherStats: false MultiFileSpray: false Stage: Stage-0 Fetch Operator limit: -1 PREHOOK: query: SELECT s.* FROM empty_bucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value PREHOOK: type: QUERY PREHOOK: Input: default@empty_bucket #### A masked pattern was here #### POSTHOOK: query: SELECT s.* FROM empty_bucket TABLESAMPLE (BUCKET 1 OUT OF 2 on key) s ORDER BY key, value POSTHOOK: type: QUERY POSTHOOK: Input: default@empty_bucket #### A masked pattern was here #### POSTHOOK: Lineage: dest1.key SIMPLE [(srcbucket)s.FieldSchema(name:key, type:int, comment:null), ] POSTHOOK: Lineage: dest1.value SIMPLE [(srcbucket)s.FieldSchema(name:value, type:string, comment:null), ]