org.apache.tez.mapreduce.hadoop

Class InputSplitInfoMem

java.lang.Object

org.apache.tez.mapreduce.hadoop.InputSplitInfoMem

All Implemented Interfaces:

InputSplitInfo

public class InputSplitInfoMem
extends Object
implements InputSplitInfo

Represents InputSplitInfo for splits generated to memory.

Since splits are generated in memory, the getSplitsMetaInfoFile and getSplitsFile are not supported.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.tez.mapreduce.hadoop.InputSplitInfo

InputSplitInfo.Type

Constructor Summary

Constructors

Constructor and Description
InputSplitInfoMem(org.apache.hadoop.mapreduce.InputSplit[] newSplits, List<org.apache.tez.dag.api.TaskLocationHint> taskLocationHints, int numTasks, org.apache.hadoop.security.Credentials credentials, org.apache.hadoop.conf.Configuration conf)
InputSplitInfoMem(org.apache.hadoop.mapred.InputSplit[] oldSplits, List<org.apache.tez.dag.api.TaskLocationHint> taskLocationHints, int numTasks, org.apache.hadoop.security.Credentials credentials, org.apache.hadoop.conf.Configuration conf)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.security.Credentials	getCredentials() Get Credentials which may be required to access the splits.
org.apache.hadoop.mapreduce.InputSplit[]	getNewFormatSplits() Get new format splits.
int	getNumTasks() Get the number of splits that were generated.
org.apache.hadoop.mapred.InputSplit[]	getOldFormatSplits() Get old format splits.
org.apache.hadoop.fs.Path	getSplitsFile() Get the path to the splits file
org.apache.hadoop.fs.Path	getSplitsMetaInfoFile() Get the path to the splits meta info file
MRRuntimeProtos.MRSplitsProto	getSplitsProto() Get the splits proto
List<org.apache.tez.dag.api.TaskLocationHint>	getTaskLocationHints() Get the TaskLocationHints for each task
InputSplitInfo.Type	getType() Get the InputSplitInfo.Type of the InputSplitInfo
boolean	holdsNewFormatSplits() Check whether the current instance is using old / new format splits

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InputSplitInfoMem

public InputSplitInfoMem(org.apache.hadoop.mapreduce.InputSplit[] newSplits,
                 List<org.apache.tez.dag.api.TaskLocationHint> taskLocationHints,
                 int numTasks,
                 org.apache.hadoop.security.Credentials credentials,
                 org.apache.hadoop.conf.Configuration conf)

InputSplitInfoMem

public InputSplitInfoMem(org.apache.hadoop.mapred.InputSplit[] oldSplits,
                 List<org.apache.tez.dag.api.TaskLocationHint> taskLocationHints,
                 int numTasks,
                 org.apache.hadoop.security.Credentials credentials,
                 org.apache.hadoop.conf.Configuration conf)

Method Detail

getTaskLocationHints

public List<org.apache.tez.dag.api.TaskLocationHint> getTaskLocationHints()

Description copied from interface: InputSplitInfo

Get the TaskLocationHints for each task

Specified by:

getTaskLocationHints in interface InputSplitInfo

getSplitsMetaInfoFile

public org.apache.hadoop.fs.Path getSplitsMetaInfoFile()

Description copied from interface: InputSplitInfo

Get the path to the splits meta info file

Specified by:

getSplitsMetaInfoFile in interface InputSplitInfo

getSplitsFile

public org.apache.hadoop.fs.Path getSplitsFile()

Description copied from interface: InputSplitInfo

Get the path to the splits file

Specified by:

getSplitsFile in interface InputSplitInfo

getNumTasks

public int getNumTasks()

Description copied from interface: InputSplitInfo

Get the number of splits that were generated. Same as number of tasks that should be run for the vertex processing these set of splits.

Specified by:

getNumTasks in interface InputSplitInfo

getType

public InputSplitInfo.Type getType()

Description copied from interface: InputSplitInfo

Get the InputSplitInfo.Type of the InputSplitInfo

Specified by:

getType in interface InputSplitInfo

getSplitsProto

public MRRuntimeProtos.MRSplitsProto getSplitsProto()

Description copied from interface: InputSplitInfo

Get the splits proto

Specified by:

getSplitsProto in interface InputSplitInfo

getCredentials

public org.apache.hadoop.security.Credentials getCredentials()

Description copied from interface: InputSplitInfo

Get Credentials which may be required to access the splits.

Specified by:

getCredentials in interface InputSplitInfo

Returns:

Credentials which may be required to access the splits.

holdsNewFormatSplits

public boolean holdsNewFormatSplits()

Description copied from interface: InputSplitInfo

Check whether the current instance is using old / new format splits

Specified by:

holdsNewFormatSplits in interface InputSplitInfo

Returns:

true if using new format splits, false otherwise

getNewFormatSplits

public org.apache.hadoop.mapreduce.InputSplit[] getNewFormatSplits()

Description copied from interface: InputSplitInfo

Get new format splits. Should only be used if the mapreduce API is being used

Specified by:

getNewFormatSplits in interface InputSplitInfo

Returns:

array of mapreduce format splits

getOldFormatSplits

public org.apache.hadoop.mapred.InputSplit[] getOldFormatSplits()

Description copied from interface: InputSplitInfo

Get old format splits. Should only be used if the mapred API is being used

Specified by:

getOldFormatSplits in interface InputSplitInfo

Returns:

array of mapred format splits