org.apache.nutch.crawl
Class CrawlDbFilter
java.lang.Object
org.apache.nutch.crawl.CrawlDbFilter
- All Implemented Interfaces:
- Closeable, JobConfigurable, Mapper<Text,CrawlDatum,Text,CrawlDatum>
public class CrawlDbFilter
- extends Object
- implements Mapper<Text,CrawlDatum,Text,CrawlDatum>
This class provides a way to separate the URL normalization
and filtering steps from the rest of CrawlDb manipulation code.
- Author:
- Andrzej Bialecki
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
URL_FILTERING
public static final String URL_FILTERING
- See Also:
- Constant Field Values
URL_NORMALIZING
public static final String URL_NORMALIZING
- See Also:
- Constant Field Values
URL_NORMALIZING_SCOPE
public static final String URL_NORMALIZING_SCOPE
- See Also:
- Constant Field Values
LOG
public static final org.apache.commons.logging.Log LOG
CrawlDbFilter
public CrawlDbFilter()
configure
public void configure(JobConf job)
- Specified by:
configure
in interface JobConfigurable
close
public void close()
- Specified by:
close
in interface Closeable
map
public void map(Text key,
CrawlDatum value,
OutputCollector<Text,CrawlDatum> output,
Reporter reporter)
throws IOException
- Specified by:
map
in interface Mapper<Text,CrawlDatum,Text,CrawlDatum>
- Throws:
IOException
Copyright © 2006 The Apache Software Foundation