org.apache.hadoop.hive.shims

Class HadoopShimsSecure.CombineFileRecordReader<K,V>

java.lang.Object

org.apache.hadoop.hive.shims.HadoopShimsSecure.CombineFileRecordReader<K,V>

All Implemented Interfaces:

org.apache.hadoop.mapred.RecordReader<K,V>

Enclosing class:

HadoopShimsSecure

public static class HadoopShimsSecure.CombineFileRecordReader<K,V>
extends Object
implements org.apache.hadoop.mapred.RecordReader<K,V>

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.mapred.RecordReader<K,V>	curReader
protected org.apache.hadoop.fs.FileSystem	fs
protected int	idx
protected boolean	isShrinked
protected org.apache.hadoop.mapred.JobConf	jc
protected long	progress
protected org.apache.hadoop.mapred.Reporter	reporter
protected Class<org.apache.hadoop.mapred.RecordReader<K,V>>	rrClass
protected Constructor<org.apache.hadoop.mapred.RecordReader<K,V>>	rrConstructor
protected long	shrinkedLength
protected org.apache.hadoop.mapred.lib.CombineFileSplit	split

Constructor Summary

Constructors

Constructor and Description
HadoopShimsSecure.CombineFileRecordReader(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.lib.CombineFileSplit split, org.apache.hadoop.mapred.Reporter reporter, Class<org.apache.hadoop.mapred.RecordReader<K,V>> rrClass) A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit.

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
K	createKey()
V	createValue()
long	getPos() Return the amount of data processed.
float	getProgress() Return progress based on the amount of data processed so far.
protected boolean	initNextRecordReader(K key) Get the record reader for the next chunk in this CombineFileSplit.
boolean	next(K key, V value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

split

protected org.apache.hadoop.mapred.lib.CombineFileSplit split

jc

protected org.apache.hadoop.mapred.JobConf jc

reporter

protected org.apache.hadoop.mapred.Reporter reporter

rrClass

protected Class<org.apache.hadoop.mapred.RecordReader<K,V>> rrClass

rrConstructor

protected Constructor<org.apache.hadoop.mapred.RecordReader<K,V>> rrConstructor

fs

protected org.apache.hadoop.fs.FileSystem fs

idx

protected int idx

progress

protected long progress

curReader

protected org.apache.hadoop.mapred.RecordReader<K,V> curReader

isShrinked

protected boolean isShrinked

shrinkedLength

protected long shrinkedLength

Constructor Detail

HadoopShimsSecure.CombineFileRecordReader

public HadoopShimsSecure.CombineFileRecordReader(org.apache.hadoop.mapred.JobConf job,
                                         org.apache.hadoop.mapred.lib.CombineFileSplit split,
                                         org.apache.hadoop.mapred.Reporter reporter,
                                         Class<org.apache.hadoop.mapred.RecordReader<K,V>> rrClass)
                                          throws IOException

A generic RecordReader that can hand out different recordReaders for each chunk in the CombineFileSplit.

Throws:

IOException

Method Detail

next

public boolean next(K key,
           V value)
             throws IOException

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

createKey

public K createKey()

Specified by:

createKey in interface org.apache.hadoop.mapred.RecordReader<K,V>

createValue

public V createValue()

Specified by:

createValue in interface org.apache.hadoop.mapred.RecordReader<K,V>

getPos

public long getPos()
            throws IOException

Return the amount of data processed.

Specified by:

getPos in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

getProgress

public float getProgress()
                  throws IOException

Return progress based on the amount of data processed so far.

Specified by:

getProgress in interface org.apache.hadoop.mapred.RecordReader<K,V>

Throws:

IOException

initNextRecordReader

protected boolean initNextRecordReader(K key)
                                throws IOException

Get the record reader for the next chunk in this CombineFileSplit.

Throws:

IOException