public static class WebGraph.OutlinkDb extends Configured implements Mapper<Text,Writable,Text,NutchWritable>, Reducer<Text,NutchWritable,Text,LinkDatum>
Modifier and Type | Field and Description |
---|---|
static String |
URL_FILTERING |
static String |
URL_NORMALIZING |
Constructor and Description |
---|
WebGraph.OutlinkDb()
Default constructor.
|
WebGraph.OutlinkDb(Configuration conf)
Configurable constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
close() |
void |
configure(JobConf conf)
Configures the OutlinkDb job.
|
void |
map(Text key,
Writable value,
OutputCollector<Text,NutchWritable> output,
Reporter reporter)
Passes through existing LinkDatum objects from an existing OutlinkDb and
maps out new LinkDatum objects from new crawls ParseData.
|
void |
reduce(Text key,
Iterator<NutchWritable> values,
OutputCollector<Text,LinkDatum> output,
Reporter reporter) |
getConf, setConf
public static final String URL_NORMALIZING
public static final String URL_FILTERING
public WebGraph.OutlinkDb()
public WebGraph.OutlinkDb(Configuration conf)
public void configure(JobConf conf)
configure
in interface JobConfigurable
public void map(Text key, Writable value, OutputCollector<Text,NutchWritable> output, Reporter reporter) throws IOException
map
in interface Mapper<Text,Writable,Text,NutchWritable>
IOException
public void reduce(Text key, Iterator<NutchWritable> values, OutputCollector<Text,LinkDatum> output, Reporter reporter) throws IOException
reduce
in interface Reducer<Text,NutchWritable,Text,LinkDatum>
IOException
public void close()
close
in interface Closeable
close
in interface AutoCloseable
Copyright © 2015 The Apache Software Foundation