org.apache.hadoop.hive.contrib.fileformat.base64

Class Base64TextInputFormat

java.lang.Object

org.apache.hadoop.hive.contrib.fileformat.base64.Base64TextInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable>, org.apache.hadoop.mapred.JobConfigurable

public class Base64TextInputFormat
extends Object
implements org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable>, org.apache.hadoop.mapred.JobConfigurable

FileInputFormat for base64 encoded text files. Each line is a base64-encoded record. The key is a LongWritable which is the offset. The value is a BytesWritable containing the base64-decoded bytes. This class accepts a configurable parameter: "base64.text.input.format.signature" The UTF-8 encoded signature will be compared with the beginning of each decoded bytes. If they don't match, the record is discarded. If they match, the signature is stripped off the data.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Base64TextInputFormat.Base64LineRecordReader Base64LineRecordReader.

Constructor Summary

Constructors

Constructor and Description
Base64TextInputFormat()

Method Summary

Modifier and Type	Method and Description
void	configure(org.apache.hadoop.mapred.JobConf job)
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable>	getRecordReader(org.apache.hadoop.mapred.InputSplit genericSplit, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
org.apache.hadoop.mapred.InputSplit[]	getSplits(org.apache.hadoop.mapred.JobConf job, int numSplits)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Base64TextInputFormat

public Base64TextInputFormat()

Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf job)

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable> getRecordReader(org.apache.hadoop.mapred.InputSplit genericSplit,
                                                                                                                                   org.apache.hadoop.mapred.JobConf job,
                                                                                                                                   org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                            throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException

getSplits

public org.apache.hadoop.mapred.InputSplit[] getSplits(org.apache.hadoop.mapred.JobConf job,
                                                       int numSplits)
                                                throws IOException

Specified by:

getSplits in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.BytesWritable>

Throws:

IOException