org.apache.nutch.net.urlnormalizer.host

Class HostURLNormalizer

java.lang.Object

org.apache.nutch.net.urlnormalizer.host.HostURLNormalizer

All Implemented Interfaces:

Configurable, URLNormalizer

public class HostURLNormalizer
extends Object
implements URLNormalizer

URL normalizer for mapping hosts to their desired form. It takes a simple text file as source in the format: example.org www.example.org mapping all URL's of example.org the the www sub-domain. It also allows for wildcards to be used to map all sub-domains to another host: *.example.org www.example.org

Field Summary

Fields inherited from interface org.apache.nutch.net.URLNormalizer

X_POINT_ID

Constructor Summary

Constructors

Constructor and Description
HostURLNormalizer()
HostURLNormalizer(String hostsFile)

Method Summary

Methods

Modifier and Type	Method and Description
Configuration	getConf()
String	normalize(String urlString, String scope)
protected String	replaceHost(String urlString, String host, String target)
void	setConf(Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HostURLNormalizer

public HostURLNormalizer()

HostURLNormalizer

public HostURLNormalizer(String hostsFile)

Method Detail

getConf

public Configuration getConf()

Specified by:

getConf in interface Configurable

setConf

public void setConf(Configuration conf)

Specified by:

setConf in interface Configurable

normalize

public String normalize(String urlString,
               String scope)
                 throws MalformedURLException

Specified by:

normalize in interface URLNormalizer

Throws:

MalformedURLException

replaceHost

protected String replaceHost(String urlString,
                 String host,
                 String target)