org.apache.hadoop.hive.ql.io

Class CombineHiveInputFormat.CombineHiveInputSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

org.apache.hadoop.mapred.lib.CombineFileSplit

org.apache.hadoop.hive.shims.HadoopShimsSecure.InputSplitShim

org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.CombineHiveInputSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable, org.apache.hadoop.mapred.InputSplit

Enclosing class:

CombineHiveInputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable>

public static class CombineHiveInputFormat.CombineHiveInputSplit
extends HadoopShimsSecure.InputSplitShim

CombineHiveInputSplit encapsulates an InputSplit with its corresponding inputFormatClassName. A CombineHiveInputSplit comprises of multiple chunks from different files. Since, they belong to a single directory, there is a single inputformat for all the chunks.

Constructor Summary

Constructors

Constructor and Description
CombineHiveInputFormat.CombineHiveInputSplit()
CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim)
CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim)
CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim, Map<String,PartitionDesc> pathToPartitionInfo)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapred.lib.CombineFileSplit	getInputSplitShim()
org.apache.hadoop.mapred.JobConf	getJob()
long	getLength()
long	getLength(int i) Returns the length of the ith Path.
long[]	getLengths() Returns an array containing the lengths of the files in the split.
String[]	getLocations() Returns all the Paths where this input-split resides.
int	getNumPaths() Returns the number of Paths in the split.
long	getOffset(int i) Returns the start offset of the ith Path.
org.apache.hadoop.fs.Path	getPath(int i) Returns the ith Path.
org.apache.hadoop.fs.Path[]	getPaths() Returns all the Paths in the split.
long[]	getStartOffsets() Returns an array containing the startoffsets of the files in the split.
String	inputFormatClassName() Returns the inputFormat class name for the i-th chunk.
void	readFields(DataInput in) Writable interface.
void	setInputFormatClassName(String inputFormatClassName)
String	toString() Prints this obejct as a string.
void	write(DataOutput out) Writable interface.

Methods inherited from class org.apache.hadoop.hive.shims.HadoopShimsSecure.InputSplitShim

getShrinkedLength, isShrinked, shrinkSplit

Methods inherited from class org.apache.hadoop.mapreduce.InputSplit

getLocationInfo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CombineHiveInputFormat.CombineHiveInputSplit

public CombineHiveInputFormat.CombineHiveInputSplit()
                                             throws IOException

Throws:

IOException

CombineHiveInputFormat.CombineHiveInputSplit

public CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim)
                                             throws IOException

Throws:

IOException

CombineHiveInputFormat.CombineHiveInputSplit

public CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.JobConf job,
                                            org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim)
                                             throws IOException

Throws:

IOException

CombineHiveInputFormat.CombineHiveInputSplit

public CombineHiveInputFormat.CombineHiveInputSplit(org.apache.hadoop.mapred.JobConf job,
                                            org.apache.hadoop.mapred.lib.CombineFileSplit inputSplitShim,
                                            Map<String,PartitionDesc> pathToPartitionInfo)
                                             throws IOException

Throws:

IOException

Method Detail

getInputSplitShim

public org.apache.hadoop.mapred.lib.CombineFileSplit getInputSplitShim()

inputFormatClassName

public String inputFormatClassName()

Returns the inputFormat class name for the i-th chunk.

setInputFormatClassName

public void setInputFormatClassName(String inputFormatClassName)

getJob

public org.apache.hadoop.mapred.JobConf getJob()

Overrides:

getJob in class org.apache.hadoop.mapred.lib.CombineFileSplit

getLength

public long getLength()

Specified by:

getLength in interface org.apache.hadoop.mapred.InputSplit

Overrides:

getLength in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getStartOffsets

public long[] getStartOffsets()

Returns an array containing the startoffsets of the files in the split.

Overrides:

getStartOffsets in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getLengths

public long[] getLengths()

Returns an array containing the lengths of the files in the split.

Overrides:

getLengths in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getOffset

public long getOffset(int i)

Returns the start offset of the i^th Path.

Overrides:

getOffset in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getLength

public long getLength(int i)

Returns the length of the i^th Path.

Overrides:

getLength in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getNumPaths

public int getNumPaths()

Returns the number of Paths in the split.

Overrides:

getNumPaths in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getPath

public org.apache.hadoop.fs.Path getPath(int i)

Returns the i^th Path.

Overrides:

getPath in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getPaths

public org.apache.hadoop.fs.Path[] getPaths()

Returns all the Paths in the split.

Overrides:

getPaths in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

getLocations

public String[] getLocations()
                      throws IOException

Returns all the Paths where this input-split resides.

Specified by:

getLocations in interface org.apache.hadoop.mapred.InputSplit

Overrides:

getLocations in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

Throws:

IOException

toString

public String toString()

Prints this obejct as a string.

Overrides:

toString in class org.apache.hadoop.mapreduce.lib.input.CombineFileSplit

readFields

public void readFields(DataInput in)
                throws IOException

Writable interface.

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class HadoopShimsSecure.InputSplitShim

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Writable interface.

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class HadoopShimsSecure.InputSplitShim

Throws:

IOException