org.apache.tez.mapreduce.output

Class MultiMROutput

java.lang.Object

org.apache.tez.runtime.api.AbstractLogicalOutput

org.apache.tez.mapreduce.output.MROutput

org.apache.tez.mapreduce.output.MultiMROutput

All Implemented Interfaces:

org.apache.tez.runtime.api.LogicalOutput, org.apache.tez.runtime.api.LogicalOutputFrameworkInterface, org.apache.tez.runtime.api.Output, org.apache.tez.runtime.api.OutputFrameworkInterface

@InterfaceAudience.Public
public class MultiMROutput
extends MROutput

MultiMROutput is an Output which allows key/values pairs to be written by a processor to different output files. It is compatible with all standard Apache Hadoop MapReduce OutputFormat implementations.

Field Summary

Fields inherited from class org.apache.tez.mapreduce.output.MROutput

committer, flushed, jobConf, outputRecordCounter

Constructor Summary

Constructors

Constructor and Description
MultiMROutput(org.apache.tez.runtime.api.OutputContext outputContext, int numPhysicalOutputs)

Method Summary

Modifier and Type	Method and Description
static org.apache.tez.mapreduce.output.MROutput.MROutputConfigBuilder	createConfigBuilder(org.apache.hadoop.conf.Configuration conf, Class<?> outputFormat, String outputPath, boolean useLazyOutputFormat) Create an MROutput.MROutputConfigBuilder
void	flush() Call this in the processor before finishing to ensure outputs that outputs have been flushed.
org.apache.tez.runtime.library.api.KeyValueWriterWithBasePath	getWriter() Get a key value write to write Map Reduce compatible output
List<org.apache.tez.runtime.api.Event>	initialize()

Methods inherited from class org.apache.tez.mapreduce.output.MROutput

abort, close, commit, createConfigBuilder, createConfigBuilder, createConfigBuilder, getOutputFileNamePrefix, getOutputName, handleEvents, initCommitter, initializeBase, isCommitRequired, start

Methods inherited from class org.apache.tez.runtime.api.AbstractLogicalOutput

getContext, getNumPhysicalOutputs

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiMROutput

public MultiMROutput(org.apache.tez.runtime.api.OutputContext outputContext,
                     int numPhysicalOutputs)

Method Detail

initialize

public List<org.apache.tez.runtime.api.Event> initialize()
                                                  throws IOException,
                                                         InterruptedException

Specified by:

initialize in interface org.apache.tez.runtime.api.OutputFrameworkInterface

Overrides:

initialize in class MROutput

Throws:

IOException

InterruptedException

createConfigBuilder

public static org.apache.tez.mapreduce.output.MROutput.MROutputConfigBuilder createConfigBuilder(org.apache.hadoop.conf.Configuration conf,
                                                                                                 Class<?> outputFormat,
                                                                                                 String outputPath,
                                                                                                 boolean useLazyOutputFormat)

Create an MROutput.MROutputConfigBuilder

Parameters:

conf - Configuration for the MROutput

outputFormat - FileInputFormat derived class

outputPath - Output path

Returns:

MROutput.MROutputConfigBuilder

getWriter

public org.apache.tez.runtime.library.api.KeyValueWriterWithBasePath getWriter()
                                                                        throws IOException

Description copied from class: MROutput

Get a key value write to write Map Reduce compatible output

Specified by:

getWriter in interface org.apache.tez.runtime.api.Output

Overrides:

getWriter in class MROutput

Throws:

IOException

flush

public void flush()
           throws IOException

Call this in the processor before finishing to ensure outputs that outputs have been flushed. Must be called before commit.

Overrides:

flush in class MROutput

Throws:

IOException