org.apache.nutch.indexer
Class DeleteDuplicates.InputFormat

java.lang.Object
  org.apache.hadoop.mapred.FileInputFormat<Text,DeleteDuplicates.IndexDoc>
      org.apache.nutch.indexer.DeleteDuplicates.InputFormat

All Implemented Interfaces:

InputFormat<Text,DeleteDuplicates.IndexDoc>

Enclosing class:

DeleteDuplicates

public static class DeleteDuplicates.InputFormat
extends FileInputFormat<Text,DeleteDuplicates.IndexDoc>

Nested Class Summary

class

DeleteDuplicates.InputFormat.DDRecordReader

Field Summary

Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

LOG

Constructor Summary

DeleteDuplicates.InputFormat()

Method Summary

RecordReader<Text,DeleteDuplicates.IndexDoc>	getRecordReader(InputSplit split, JobConf job, Reporter reporter) Return each index as a split.
InputSplit[]	getSplits(JobConf job, int numSplits) Return each index as a split.

Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

addInputPath, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, isSplitable, listStatus, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DeleteDuplicates.InputFormat

public DeleteDuplicates.InputFormat()

Method Detail

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws IOException

Return each index as a split.

Specified by:

getSplits in interface InputFormat<Text,DeleteDuplicates.IndexDoc>

Overrides:

getSplits in class FileInputFormat<Text,DeleteDuplicates.IndexDoc>

Throws:

IOException

getRecordReader

public RecordReader<Text,DeleteDuplicates.IndexDoc> getRecordReader(InputSplit split,
                                                                    JobConf job,
                                                                    Reporter reporter)
                                                             throws IOException

Return each index as a split.

Specified by:

getRecordReader in interface InputFormat<Text,DeleteDuplicates.IndexDoc>

Specified by:

getRecordReader in class FileInputFormat<Text,DeleteDuplicates.IndexDoc>

Throws:

IOException