org.apache.hive.hcatalog.mapreduce

Class HCatBaseOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

org.apache.hive.hcatalog.mapreduce.HCatBaseOutputFormat

Direct Known Subclasses:

HCatOutputFormat

public abstract class HCatBaseOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

Constructor Summary

Constructors

Constructor and Description
HCatBaseOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Check for validity of the output-specification for the job.
protected static void	configureOutputStorageHandler(org.apache.hadoop.mapreduce.JobContext context, OutputJobInfo jobInfo, Map<String,String> fullPartSpec) Configure the output storage handler, with allowing specification of partvals from which it picks the dynamic partvals
static OutputJobInfo	getJobInfo(org.apache.hadoop.conf.Configuration conf) Gets the HCatOuputJobInfo object by reading the Configuration and deserializing the string.
protected org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>	getOutputFormat(org.apache.hadoop.mapreduce.JobContext context) Gets the output format instance.
static HCatSchema	getTableSchema(org.apache.hadoop.conf.Configuration conf) Gets the table schema for the table specified in the HCatOutputFormat.setOutput call on the specified job context.
protected static void	setPartDetails(OutputJobInfo jobInfo, HCatSchema schema, Map<String,String> partMap)

Methods inherited from class org.apache.hadoop.mapreduce.OutputFormat

getOutputCommitter, getRecordWriter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HCatBaseOutputFormat

public HCatBaseOutputFormat()

Method Detail

getTableSchema

public static HCatSchema getTableSchema(org.apache.hadoop.conf.Configuration conf)
                                 throws IOException

Gets the table schema for the table specified in the HCatOutputFormat.setOutput call on the specified job context.

Parameters:

conf - the Configuration object

Returns:

the table schema

Throws:

IOException - if HCatOutputFormat.setOutput has not been called for the passed context

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Check for validity of the output-specification for the job.

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>

Parameters:

context - information about the job

Throws:

IOException - when output should not be attempted

InterruptedException

getOutputFormat

protected org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord> getOutputFormat(org.apache.hadoop.mapreduce.JobContext context)
                                                                                                                   throws IOException

Gets the output format instance.

Parameters:

context - the job context

Returns:

the output format instance

Throws:

IOException

getJobInfo

public static OutputJobInfo getJobInfo(org.apache.hadoop.conf.Configuration conf)
                                throws IOException

Gets the HCatOuputJobInfo object by reading the Configuration and deserializing the string. If InputJobInfo is not present in the configuration, throws an exception since that means HCatOutputFormat.setOutput has not been called.

Parameters:

conf - the job Configuration object

Returns:

the OutputJobInfo object

Throws:

IOException - the IO exception

configureOutputStorageHandler

protected static void configureOutputStorageHandler(org.apache.hadoop.mapreduce.JobContext context,
                                 OutputJobInfo jobInfo,
                                 Map<String,String> fullPartSpec)
                                             throws IOException

Configure the output storage handler, with allowing specification of partvals from which it picks the dynamic partvals

Parameters:

context - the job context

jobInfo - the output job info

fullPartSpec -

Throws:

IOException

setPartDetails

protected static void setPartDetails(OutputJobInfo jobInfo,
                  HCatSchema schema,
                  Map<String,String> partMap)
                              throws HCatException,
                                     IOException

Throws:

HCatException

IOException