org.apache.nutch.fetcher
Class OldFetcher
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.nutch.fetcher.OldFetcher
- All Implemented Interfaces:
- Configurable, JobConfigurable, MapRunnable<WritableComparable,Writable,Text,NutchWritable>, Tool
public class OldFetcher
- extends Configured
- implements Tool, MapRunnable<WritableComparable,Writable,Text,NutchWritable>
The fetcher. Most of the work is done by plugins.
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
public static final org.slf4j.Logger LOG
PERM_REFRESH_TIME
public static final int PERM_REFRESH_TIME
- See Also:
- Constant Field Values
CONTENT_REDIR
public static final String CONTENT_REDIR
- See Also:
- Constant Field Values
PROTOCOL_REDIR
public static final String PROTOCOL_REDIR
- See Also:
- Constant Field Values
OldFetcher
public OldFetcher()
OldFetcher
public OldFetcher(Configuration conf)
configure
public void configure(JobConf job)
- Specified by:
configure
in interface JobConfigurable
close
public void close()
isParsing
public static boolean isParsing(Configuration conf)
isStoringContent
public static boolean isStoringContent(Configuration conf)
run
public void run(RecordReader<WritableComparable,Writable> input,
OutputCollector<Text,NutchWritable> output,
Reporter reporter)
throws IOException
- Specified by:
run
in interface MapRunnable<WritableComparable,Writable,Text,NutchWritable>
- Throws:
IOException
fetch
public void fetch(Path segment,
int threads)
throws IOException
- Throws:
IOException
main
public static void main(String[] args)
throws Exception
- Run the fetcher.
- Throws:
Exception
run
public int run(String[] args)
throws Exception
- Specified by:
run
in interface Tool
- Throws:
Exception
Copyright © 2011 The Apache Software Foundation