org.apache.nutch.scoring.webgraph

Class NodeDumper.Dumper

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.scoring.webgraph.NodeDumper.Dumper

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.Text,Node,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>, org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>

Enclosing class:

NodeDumper

public static class NodeDumper.Dumper
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.Text,Node,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>, org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>

Outputs the hosts or domains with an associated value. This value consists of either the number of inlinks, the number of outlinks or the score. The computed value is then either the sum of all parts or the top value.

Constructor Summary

Constructors

Constructor and Description
NodeDumper.Dumper()

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.hadoop.mapred.JobConf conf)
void	map(org.apache.hadoop.io.Text key, Node node, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable> output, org.apache.hadoop.mapred.Reporter reporter) Outputs the host or domain as key for this record and numInlinks, numOutlinks or score as the value.
void	reduce(org.apache.hadoop.io.Text key, Iterator<org.apache.hadoop.io.FloatWritable> values, org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable> output, org.apache.hadoop.mapred.Reporter reporter) Outputs either the sum or the top value for this record.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NodeDumper.Dumper

public NodeDumper.Dumper()

Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf conf)

Specified by:

configure in interface org.apache.hadoop.mapred.JobConfigurable

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

map

public void map(org.apache.hadoop.io.Text key,
       Node node,
       org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable> output,
       org.apache.hadoop.mapred.Reporter reporter)
         throws IOException

Outputs the host or domain as key for this record and numInlinks, numOutlinks or score as the value.

Specified by:

map in interface org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.Text,Node,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>

Throws:

IOException

reduce

public void reduce(org.apache.hadoop.io.Text key,
          Iterator<org.apache.hadoop.io.FloatWritable> values,
          org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable> output,
          org.apache.hadoop.mapred.Reporter reporter)
            throws IOException

Outputs either the sum or the top value for this record.

Specified by:

reduce in interface org.apache.hadoop.mapred.Reducer<org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.FloatWritable>

Throws:

IOException