Package | Description |
---|---|
org.apache.nutch.fetcher |
The Nutch robot.
|
org.apache.nutch.indexer |
Maintain Lucene full-text indexes.
|
org.apache.nutch.metadata |
A Multi-valued Metadata container, and set
of constant fields for Nutch Metadata.
|
org.apache.nutch.scoring.webgraph | |
org.apache.nutch.segment | |
org.apache.nutch.tools.arc |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.mapred.RecordWriter<org.apache.hadoop.io.Text,NutchWritable> |
FetcherOutputFormat.getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.mapred.JobConf job,
String name,
org.apache.hadoop.util.Progressable progress) |
Modifier and Type | Method and Description |
---|---|
void |
Fetcher.run(org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.Text,CrawlDatum> input,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> output,
org.apache.hadoop.mapred.Reporter reporter) |
void |
OldFetcher.run(org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.WritableComparable<?>,org.apache.hadoop.io.Writable> input,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> output,
org.apache.hadoop.mapred.Reporter reporter) |
Modifier and Type | Method and Description |
---|---|
void |
IndexerMapReduce.map(org.apache.hadoop.io.Text key,
org.apache.hadoop.io.Writable value,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> output,
org.apache.hadoop.mapred.Reporter reporter) |
void |
IndexerMapReduce.reduce(org.apache.hadoop.io.Text key,
Iterator<NutchWritable> values,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchIndexAction> output,
org.apache.hadoop.mapred.Reporter reporter) |
Modifier and Type | Class and Description |
---|---|
class |
MetaWrapper
This is a simple decorator that adds metadata to any Writable-s that can be
serialized by NutchWritable.
|
Modifier and Type | Method and Description |
---|---|
void |
WebGraph.OutlinkDb.map(org.apache.hadoop.io.Text key,
org.apache.hadoop.io.Writable value,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> output,
org.apache.hadoop.mapred.Reporter reporter)
Passes through existing LinkDatum objects from an existing OutlinkDb and
maps out new LinkDatum objects from new crawls ParseData.
|
void |
WebGraph.OutlinkDb.reduce(org.apache.hadoop.io.Text key,
Iterator<NutchWritable> values,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,LinkDatum> output,
org.apache.hadoop.mapred.Reporter reporter) |
Modifier and Type | Method and Description |
---|---|
void |
SegmentReader.InputCompatMapper.map(org.apache.hadoop.io.WritableComparable<?> key,
org.apache.hadoop.io.Writable value,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> collector,
org.apache.hadoop.mapred.Reporter reporter) |
void |
SegmentReader.reduce(org.apache.hadoop.io.Text key,
Iterator<NutchWritable> values,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> output,
org.apache.hadoop.mapred.Reporter reporter) |
Modifier and Type | Method and Description |
---|---|
void |
ArcSegmentCreator.map(org.apache.hadoop.io.Text key,
org.apache.hadoop.io.BytesWritable bytes,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,NutchWritable> output,
org.apache.hadoop.mapred.Reporter reporter)
Runs the Map job to translate an arc record into output for Nutch
segments.
|
Copyright © 2014 The Apache Software Foundation