org.apache.nutch.scoring.webgraph

Class LinkDumper

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.nutch.scoring.webgraph.LinkDumper

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class LinkDumper
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

The LinkDumper tool creates a database of node to inlink information that can be read using the nested Reader class. This allows the inlink and scoring state of a single url to be reviewed quickly to determine why a given url is ranking a certain way. This tool is to be used with the LinkRank analysis.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LinkDumper.Inverter Inverts outlinks from the WebGraph to inlinks and attaches node information.
static class	LinkDumper.LinkNode Bean class which holds url to node information.
static class	LinkDumper.LinkNodes Writable class which holds an array of LinkNode objects.
static class	LinkDumper.Merger Merges LinkNode objects into a single array value per url.
static class	LinkDumper.Reader Reader class which will print out the url and all of its inlinks to system out.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DUMP_DIR
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
LinkDumper()

Method Summary

Methods

Modifier and Type	Method and Description
void	dumpLinks(org.apache.hadoop.fs.Path webGraphDb) Runs the inverter and merger jobs of the LinkDumper tool to create the url to inlink node database.
static void	main(String[] args)
int	run(String[] args) Runs the LinkDumper tool.

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

LOG

public static final org.slf4j.Logger LOG

DUMP_DIR

public static final String DUMP_DIR

See Also:

Constant Field Values

Constructor Detail

LinkDumper

public LinkDumper()

Method Detail

dumpLinks

public void dumpLinks(org.apache.hadoop.fs.Path webGraphDb)
               throws IOException

Runs the inverter and merger jobs of the LinkDumper tool to create the url to inlink node database.

Throws:

IOException

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

run

public int run(String[] args)
        throws Exception

Runs the LinkDumper tool. This simply creates the database, to read the values the nested Reader tool must be used.

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception