A B C D E F G H I J K L M N O P R S T U V W X Z _

A

AbstractFetchSchedule - Class in org.apache.nutch.crawl
This class provides common methods for implementations of FetchSchedule.
AbstractFetchSchedule() - Constructor for class org.apache.nutch.crawl.AbstractFetchSchedule
 
AbstractFetchSchedule(Configuration) - Constructor for class org.apache.nutch.crawl.AbstractFetchSchedule
 
AbstractTestbedHandler - Class in org.apache.nutch.tools.proxy
 
AbstractTestbedHandler() - Constructor for class org.apache.nutch.tools.proxy.AbstractTestbedHandler
 
accept() - Method in class org.apache.nutch.urlfilter.api.RegexRule
Return if this rule is used for filtering-in or out.
acceptLanguage - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The "Accept-Language" request header value.
ACCESS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Access denied - authorization required, but missing/incorrect.
AdaptiveFetchSchedule - Class in org.apache.nutch.crawl
This class implements an adaptive re-fetch algorithm.
AdaptiveFetchSchedule() - Constructor for class org.apache.nutch.crawl.AdaptiveFetchSchedule
 
add(String) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Add ngrams from a token to this profile
add(StringBuffer) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Add ngrams from a single word to this profile
add(Inlink) - Method in class org.apache.nutch.crawl.Inlinks
 
add(Inlinks) - Method in class org.apache.nutch.crawl.Inlinks
 
add(String, Object) - Method in class org.apache.nutch.indexer.NutchDocument
 
add(Object) - Method in class org.apache.nutch.indexer.NutchField
 
add(String, String) - Method in class org.apache.nutch.metadata.Metadata
Add a metadata name/value mapping.
add(String, String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
 
addAttribute(String, String) - Method in class org.apache.nutch.plugin.Extension
Adds a attribute and is only used until model creation at plugin system start up.
addClassToConf(Configuration, Class<? extends NutchIndexWriter>) - Static method in class org.apache.nutch.indexer.NutchIndexWriterFactory
 
addClue(String, String, int) - Method in class org.apache.nutch.util.EncodingDetector
 
addClue(String, String) - Method in class org.apache.nutch.util.EncodingDetector
 
addDependency(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
Adds a dependency
addExportedLibRelative(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
Adds a exported library with a relative path to the plugin directory.
addExtension(Extension) - Method in class org.apache.nutch.plugin.ExtensionPoint
Install a coresponding extension to this extension point.
addExtension(Extension) - Method in class org.apache.nutch.plugin.PluginDescriptor
Adds a extension.
addExtensionPoint(ExtensionPoint) - Method in class org.apache.nutch.plugin.PluginDescriptor
Adds a extension point.
addMeta(String, String) - Method in class org.apache.nutch.metadata.MetaWrapper
Add metadata.
addMyHeader(HttpServletResponse, String, String) - Method in class org.apache.nutch.tools.proxy.AbstractTestbedHandler
 
addNotExportedLibRelative(String) - Method in class org.apache.nutch.plugin.PluginDescriptor
Adds a not exported library with a plugin directory relative path.
addPatternBackward(String) - Method in class org.apache.nutch.util.TrieStringMatcher
Adds any necessary nodes to the trie so that the given String can be decoded in reverse and the first character is represented by a terminal node.
addPatternForward(String) - Method in class org.apache.nutch.util.TrieStringMatcher
Adds any necessary nodes to the trie so that the given String can be decoded and the last character is represented by a terminal node.
addTiming(String, String, long) - Method in class org.apache.nutch.tools.Benchmark.BenchmarkResults
 
addUrlFeatures(NutchDocument, String) - Method in class org.creativecommons.nutch.CCIndexingFilter
Add the features represented by a license URL.
analyze(StringBuilder) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Analyze a piece of text
analyze(Path) - Method in class org.apache.nutch.scoring.webgraph.LinkRank
Runs the complete link analysis job.
APPLICATION_NAME - Static variable in interface org.apache.nutch.metadata.Office
 
ArcInputFormat - Class in org.apache.nutch.tools.arc
A input format the reads arc files.
ArcInputFormat() - Constructor for class org.apache.nutch.tools.arc.ArcInputFormat
 
ArcRecordReader - Class in org.apache.nutch.tools.arc
The ArchRecordReader class provides a record reader which reads records from arc files.
ArcRecordReader(Configuration, FileSplit) - Constructor for class org.apache.nutch.tools.arc.ArcRecordReader
Constructor that sets the configuration and file split.
ArcSegmentCreator - Class in org.apache.nutch.tools.arc
The ArcSegmentCreator is a replacement for fetcher that will take arc files as input and produce a nutch segment as output.
ArcSegmentCreator() - Constructor for class org.apache.nutch.tools.arc.ArcSegmentCreator
 
ArcSegmentCreator(Configuration) - Constructor for class org.apache.nutch.tools.arc.ArcSegmentCreator
Constructor that sets the job configuration.
AUTHOR - Static variable in interface org.apache.nutch.metadata.Office
 
autoDetectClues(Content, boolean) - Method in class org.apache.nutch.util.EncodingDetector
 
AutomatonURLFilter - Class in org.apache.nutch.urlfilter.automaton
RegexURLFilterBase implementation based on the dk.brics.automaton Finite-State Automata for JavaTM.
AutomatonURLFilter() - Constructor for class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
AutomatonURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
autoResolveContentType(String, String, byte[]) - Method in class org.apache.nutch.util.MimeUtil
A facade interface to trying all the possible mime type resolution strategies available within Tika.

B

BasicIndexingFilter - Class in org.apache.nutch.indexer.basic
Adds basic searchable fields to a document.
BasicIndexingFilter() - Constructor for class org.apache.nutch.indexer.basic.BasicIndexingFilter
 
Benchmark - Class in org.apache.nutch.tools
 
Benchmark() - Constructor for class org.apache.nutch.tools.Benchmark
 
benchmark(int, int, int, int, long, boolean, String) - Method in class org.apache.nutch.tools.Benchmark
 
Benchmark.BenchmarkResults - Class in org.apache.nutch.tools
 
Benchmark.BenchmarkResults() - Constructor for class org.apache.nutch.tools.Benchmark.BenchmarkResults
 
BLOCKED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Thread was blocked http.max.delays times during fetching.
BlockedException - Exception in org.apache.nutch.protocol.http.api
 
BlockedException(String) - Constructor for exception org.apache.nutch.protocol.http.api.BlockedException
 
BOOST_FIELD - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
BUFFER_SIZE - Static variable in class org.apache.nutch.protocol.http.api.HttpBase
 

C

CACHING_FORBIDDEN_ALL - Static variable in interface org.apache.nutch.metadata.Nutch
Don't show either original forbidden content or summaries.
CACHING_FORBIDDEN_CONTENT - Static variable in interface org.apache.nutch.metadata.Nutch
Don't show original forbidden content, but show summaries.
CACHING_FORBIDDEN_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
Sites may request that search engines don't provide access to cached documents.
CACHING_FORBIDDEN_NONE - Static variable in interface org.apache.nutch.metadata.Nutch
Show both original forbidden content and summaries (default).
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.MD5Signature
 
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.Signature
 
calculate(Content, Parse) - Method in class org.apache.nutch.crawl.TextProfileSignature
 
calculateLastFetchTime(CrawlDatum) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
This method return the last fetch time of the CrawlDatum
calculateLastFetchTime(CrawlDatum) - Method in interface org.apache.nutch.crawl.FetchSchedule
Calculates last fetch time of the given CrawlDatum.
CCIndexingFilter - Class in org.creativecommons.nutch
Adds basic searchable fields to a document.
CCIndexingFilter() - Constructor for class org.creativecommons.nutch.CCIndexingFilter
 
CCParseFilter - Class in org.creativecommons.nutch
Adds metadata identifying the Creative Commons license used, if any.
CCParseFilter() - Constructor for class org.creativecommons.nutch.CCParseFilter
 
CCParseFilter.Walker - Class in org.creativecommons.nutch
Walks DOM tree, looking for RDF in comments and licenses in anchors.
CHAR_ENCODING_FOR_CONVERSION - Static variable in interface org.apache.nutch.metadata.Nutch
 
CHARACTER_COUNT - Static variable in interface org.apache.nutch.metadata.Office
 
CHECK_BLOCKING - Static variable in interface org.apache.nutch.protocol.Protocol
Property name.
CHECK_ROBOTS - Static variable in interface org.apache.nutch.protocol.Protocol
Property name.
checkClientTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
 
checkOutputSpecs(FileSystem, JobConf) - Method in class org.apache.nutch.fetcher.FetcherOutputFormat
 
checkOutputSpecs(FileSystem, JobConf) - Method in class org.apache.nutch.parse.ParseOutputFormat
 
checkServerTrusted(X509Certificate[], String) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
 
children - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
 
childrenList - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
 
chooseRepr(String, String, boolean) - Static method in class org.apache.nutch.util.URLUtil
Given two urls, a src and a destination of a redirect, it returns the representative url.
CircularDependencyException - Exception in org.apache.nutch.plugin
CircularDependencyException will be thrown if a circular dependency is detected.
CircularDependencyException(Throwable) - Constructor for exception org.apache.nutch.plugin.CircularDependencyException
 
CircularDependencyException(String) - Constructor for exception org.apache.nutch.plugin.CircularDependencyException
 
cleanMimeType(String) - Static method in class org.apache.nutch.util.MimeUtil
Cleans a MimeType name by removing out the actual MimeType, from a string of the form:
clear() - Method in class org.apache.nutch.crawl.Inlinks
 
clear() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
clear() - Method in class org.apache.nutch.metadata.Metadata
Remove all mappings from metadata.
clearClues() - Method in class org.apache.nutch.util.EncodingDetector
Clears all clues.
Client - Class in org.apache.nutch.protocol.ftp
Client.java encapsulates functionalities necessary for nutch to get dir list and retrieve file from an FTP server.
Client() - Constructor for class org.apache.nutch.protocol.ftp.Client
 
clone() - Method in class org.apache.nutch.crawl.CrawlDatum
 
close() - Method in class org.apache.nutch.crawl.CrawlDbFilter
 
close() - Method in class org.apache.nutch.crawl.CrawlDbMerger.Merger
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader
 
close(Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatCombiner
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
 
close() - Method in class org.apache.nutch.crawl.CrawlDbReducer
 
close() - Method in class org.apache.nutch.crawl.Generator.Selector
 
close() - Method in class org.apache.nutch.crawl.Injector.InjectMapper
 
close() - Method in class org.apache.nutch.crawl.Injector.InjectReducer
 
close() - Method in class org.apache.nutch.crawl.LinkDb
 
close() - Method in class org.apache.nutch.crawl.LinkDbFilter
 
close() - Method in class org.apache.nutch.crawl.LinkDbMerger
 
close() - Method in class org.apache.nutch.crawl.LinkDbReader
 
close() - Method in class org.apache.nutch.crawl.URLPartitioner
 
close() - Method in class org.apache.nutch.fetcher.Fetcher
 
close() - Method in class org.apache.nutch.fetcher.OldFetcher
 
close() - Method in class org.apache.nutch.indexer.IndexerMapReduce
 
close() - Method in interface org.apache.nutch.indexer.NutchIndexWriter
 
close() - Method in class org.apache.nutch.indexer.solr.SolrClean.DBFilter
 
close() - Method in class org.apache.nutch.indexer.solr.SolrClean.SolrDeleter
 
close() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
close() - Method in class org.apache.nutch.indexer.solr.SolrWriter
 
close() - Method in class org.apache.nutch.parse.ParseSegment
 
close() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
 
close() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
 
close() - Method in class org.apache.nutch.scoring.webgraph.LinkRank
 
close() - Method in class org.apache.nutch.scoring.webgraph.Loops.Finalizer
 
close() - Method in class org.apache.nutch.scoring.webgraph.Loops.Initializer
 
close() - Method in class org.apache.nutch.scoring.webgraph.Loops.Looper
 
close() - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
 
close() - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
 
close() - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
 
close() - Method in class org.apache.nutch.segment.SegmentMerger
 
close() - Method in class org.apache.nutch.segment.SegmentReader
 
close() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Closes the record reader resources.
close() - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
 
close() - Method in class org.apache.nutch.tools.CrawlDBScanner
 
closeReaders(SequenceFile.Reader[]) - Static method in class org.apache.nutch.util.FSUtils
Closes a group of SequenceFile readers.
closeReaders(MapFile.Reader[]) - Static method in class org.apache.nutch.util.FSUtils
Closes a group of MapFile readers.
CommandRunner - Class in org.apache.nutch.util
 
CommandRunner() - Constructor for class org.apache.nutch.util.CommandRunner
 
COMMENTS - Static variable in interface org.apache.nutch.metadata.Office
 
COMMIT_SIZE - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.CrawlDatum.Comparator
 
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.Generator.DecreasingFloatComparator
Compares two FloatWritables decreasing.
compare(WritableComparable, WritableComparable) - Method in class org.apache.nutch.crawl.Generator.HashComparator
 
compare(byte[], int, int, byte[], int, int) - Method in class org.apache.nutch.crawl.Generator.HashComparator
 
compare(Object, Object) - Method in class org.apache.nutch.crawl.SignatureComparator
 
compareTo(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
Sort by decreasing score.
compareTo(TrieStringMatcher.TrieNode) - Method in class org.apache.nutch.util.TrieStringMatcher.TrieNode
 
conf - Variable in class org.apache.nutch.crawl.Signature
 
conf - Variable in class org.apache.nutch.plugin.Plugin
 
conf - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbFilter
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbMerger.Merger
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatCombiner
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
 
configure(JobConf) - Method in class org.apache.nutch.crawl.CrawlDbReducer
 
configure(JobConf) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater
 
configure(JobConf) - Method in class org.apache.nutch.crawl.Generator.Selector
 
configure(JobConf) - Method in class org.apache.nutch.crawl.Injector.InjectMapper
 
configure(JobConf) - Method in class org.apache.nutch.crawl.Injector.InjectReducer
 
configure(JobConf) - Method in class org.apache.nutch.crawl.LinkDb
 
configure(JobConf) - Method in class org.apache.nutch.crawl.LinkDbFilter
 
configure(JobConf) - Method in class org.apache.nutch.crawl.LinkDbMerger
 
configure(JobConf) - Method in class org.apache.nutch.crawl.URLPartitioner
 
configure(JobConf) - Method in class org.apache.nutch.fetcher.Fetcher
 
configure(JobConf) - Method in class org.apache.nutch.fetcher.OldFetcher
 
configure(JobConf) - Method in class org.apache.nutch.indexer.IndexerMapReduce
 
configure(JobConf) - Method in class org.apache.nutch.indexer.solr.SolrClean.DBFilter
 
configure(JobConf) - Method in class org.apache.nutch.indexer.solr.SolrClean.SolrDeleter
 
configure(JobConf) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
configure(JobConf) - Method in class org.apache.nutch.parse.ParseSegment
 
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
 
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
 
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.Loops.Finalizer
Configures the job.
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.Loops.Initializer
Configure the job.
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.Loops.Looper
Configure the job.
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
Configures the job, sets the flag for type of content and the topN number if any.
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
 
configure(JobConf) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
Configures the OutlinkDb job.
configure(JobConf) - Method in class org.apache.nutch.segment.SegmentMerger
 
configure(JobConf) - Method in class org.apache.nutch.segment.SegmentReader
 
configure(JobConf) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
Configures the job.
configure(JobConf) - Method in class org.apache.nutch.tools.CrawlDBScanner
 
configure(JobConf) - Method in class org.apache.nutch.tools.FreeGenerator.FG
 
configure(JobConf) - Method in class org.apache.nutch.util.domain.DomainStatistics
 
containsKey(Writable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
containsValue(Writable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
Content - Class in org.apache.nutch.protocol
 
Content() - Constructor for class org.apache.nutch.protocol.Content
 
Content(String, String, byte[], String, Metadata, Configuration) - Constructor for class org.apache.nutch.protocol.Content
 
CONTENT_DISPOSITION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_ENCODING - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_LANGUAGE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_LENGTH - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_LOCATION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_MD5 - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
CONTENT_REDIR - Static variable in class org.apache.nutch.fetcher.Fetcher
 
CONTENT_REDIR - Static variable in class org.apache.nutch.fetcher.OldFetcher
 
CONTENT_TYPE - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
ContentAsTextInputFormat - Class in org.apache.nutch.segment
An input format that takes Nutch Content objects and converts them to text while converting newline endings to spaces.
ContentAsTextInputFormat() - Constructor for class org.apache.nutch.segment.ContentAsTextInputFormat
 
CONTRIBUTOR - Static variable in interface org.apache.nutch.metadata.DublinCore
An entity responsible for making contributions to the content of the resource.
COVERAGE - Static variable in interface org.apache.nutch.metadata.DublinCore
The extent or scope of the content of the resource.
Crawl - Class in org.apache.nutch.crawl
 
Crawl() - Constructor for class org.apache.nutch.crawl.Crawl
 
CrawlDatum - Class in org.apache.nutch.crawl
 
CrawlDatum() - Constructor for class org.apache.nutch.crawl.CrawlDatum
 
CrawlDatum(int, int) - Constructor for class org.apache.nutch.crawl.CrawlDatum
 
CrawlDatum(int, int, float) - Constructor for class org.apache.nutch.crawl.CrawlDatum
 
CrawlDatum.Comparator - Class in org.apache.nutch.crawl
A Comparator optimized for CrawlDatum.
CrawlDatum.Comparator() - Constructor for class org.apache.nutch.crawl.CrawlDatum.Comparator
 
CrawlDb - Class in org.apache.nutch.crawl
This class takes the output of the fetcher and updates the crawldb accordingly.
CrawlDb() - Constructor for class org.apache.nutch.crawl.CrawlDb
 
CrawlDb(Configuration) - Constructor for class org.apache.nutch.crawl.CrawlDb
 
CRAWLDB_ADDITIONS_ALLOWED - Static variable in class org.apache.nutch.crawl.CrawlDb
 
CrawlDbFilter - Class in org.apache.nutch.crawl
This class provides a way to separate the URL normalization and filtering steps from the rest of CrawlDb manipulation code.
CrawlDbFilter() - Constructor for class org.apache.nutch.crawl.CrawlDbFilter
 
CrawlDbMerger - Class in org.apache.nutch.crawl
This tool merges several CrawlDb-s into one, optionally filtering URLs through the current URLFilters, to skip prohibited pages.
CrawlDbMerger() - Constructor for class org.apache.nutch.crawl.CrawlDbMerger
 
CrawlDbMerger(Configuration) - Constructor for class org.apache.nutch.crawl.CrawlDbMerger
 
CrawlDbMerger.Merger - Class in org.apache.nutch.crawl
 
CrawlDbMerger.Merger() - Constructor for class org.apache.nutch.crawl.CrawlDbMerger.Merger
 
CrawlDbReader - Class in org.apache.nutch.crawl
Read utility for the CrawlDB.
CrawlDbReader() - Constructor for class org.apache.nutch.crawl.CrawlDbReader
 
CrawlDbReader.CrawlDatumCsvOutputFormat - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDatumCsvOutputFormat() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat
 
CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter(DataOutputStream) - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
 
CrawlDbReader.CrawlDbStatCombiner - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDbStatCombiner() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatCombiner
 
CrawlDbReader.CrawlDbStatMapper - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDbStatMapper() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
 
CrawlDbReader.CrawlDbStatReducer - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDbStatReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
 
CrawlDbReader.CrawlDbTopNMapper - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDbTopNMapper() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
 
CrawlDbReader.CrawlDbTopNReducer - Class in org.apache.nutch.crawl
 
CrawlDbReader.CrawlDbTopNReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
 
CrawlDbReducer - Class in org.apache.nutch.crawl
Merge new page entries with existing entries.
CrawlDbReducer() - Constructor for class org.apache.nutch.crawl.CrawlDbReducer
 
CrawlDBScanner - Class in org.apache.nutch.tools
Dumps all the entries matching a regular expression on their URL.
CrawlDBScanner() - Constructor for class org.apache.nutch.tools.CrawlDBScanner
 
CrawlDBScanner(Configuration) - Constructor for class org.apache.nutch.tools.CrawlDBScanner
 
create(String, InputStream, String) - Static method in class org.apache.nutch.analysis.lang.NGramProfile
Create a new Language profile from (preferably quite large) text file
create() - Static method in class org.apache.nutch.util.NutchConfiguration
Create a Configuration for Nutch.
create(boolean, Properties) - Static method in class org.apache.nutch.util.NutchConfiguration
Create a Configuration from supplied properties.
createJob(Configuration, Path) - Static method in class org.apache.nutch.crawl.CrawlDb
 
createKey() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Creates a new instance of the Text object for the key.
createLockFile(FileSystem, Path, boolean) - Static method in class org.apache.nutch.util.LockUtil
Create a lock file.
createMergeJob(Configuration, Path, boolean, boolean) - Static method in class org.apache.nutch.crawl.CrawlDbMerger
 
createMergeJob(Configuration, Path, boolean, boolean) - Static method in class org.apache.nutch.crawl.LinkDbMerger
 
createParseResult(String, Parse) - Static method in class org.apache.nutch.parse.ParseResult
Convenience method for obtaining ParseResult from a single Parse output.
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Creates a new RegexRule.
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
createRule(boolean, String) - Method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
createSegments(Path, Path) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
Creates the arc files to segments job.
createSocket(String, int, InetAddress, int) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
 
createSocket(String, int, InetAddress, int, HttpConnectionParams) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
Attempts to get a new socket connection to the given host within the given time limit.
createSocket(String, int) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
 
createSocket(Socket, String, int, boolean) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
 
createValue() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Creates a new instance of the BytesWritable object for the key
createWebGraph(Path, Path[]) - Method in class org.apache.nutch.scoring.webgraph.WebGraph
Creates the three different WebGraph databases, Outlinks, Inlinks, and Node.
CreativeCommons - Interface in org.apache.nutch.metadata
A collection of Creative Commons properties names.
CREATOR - Static variable in interface org.apache.nutch.metadata.DublinCore
An entity primarily responsible for making the content of the resource.
CSV_FORMAT - Static variable in class org.apache.nutch.crawl.CrawlDbReader
 
CURRENT_NAME - Static variable in class org.apache.nutch.crawl.CrawlDb
 
CURRENT_NAME - Static variable in class org.apache.nutch.crawl.LinkDb
 

D

DATE - Static variable in interface org.apache.nutch.metadata.DublinCore
A date associated with an event in the life cycle of the resource.
datum - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
 
debug - Variable in class org.apache.nutch.tools.proxy.AbstractTestbedHandler
 
dedup(String) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
DEFAULT_BOOST - Static variable in class org.apache.nutch.util.domain.DomainSuffix
 
DEFAULT_DELAY - Static variable in class org.apache.nutch.tools.proxy.DelayHandler
 
DEFAULT_PLUGIN - Static variable in class org.apache.nutch.parse.ParserFactory
Wildcard for default plugins.
DEFAULT_STATUS - Static variable in class org.apache.nutch.util.domain.DomainSuffix
 
DefaultFetchSchedule - Class in org.apache.nutch.crawl
This class implements the default re-fetch schedule.
DefaultFetchSchedule() - Constructor for class org.apache.nutch.crawl.DefaultFetchSchedule
 
defaultInterval - Variable in class org.apache.nutch.crawl.AbstractFetchSchedule
 
deflate(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
Returns a deflated copy of the input array.
DeflateUtils - Class in org.apache.nutch.util
A collection of utility methods for working on deflated data.
DeflateUtils() - Constructor for class org.apache.nutch.util.DeflateUtils
 
DelayHandler - Class in org.apache.nutch.tools.proxy
 
DelayHandler(int) - Constructor for class org.apache.nutch.tools.proxy.DelayHandler
 
delete(String, String) - Method in class org.apache.nutch.indexer.solr.SolrClean
 
DESCRIPTION - Static variable in interface org.apache.nutch.metadata.DublinCore
An account of the content of the resource.
DIGEST_FIELD - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
DIR_NAME - Static variable in class org.apache.nutch.parse.ParseData
 
DIR_NAME - Static variable in class org.apache.nutch.parse.ParseText
 
DIR_NAME - Static variable in class org.apache.nutch.protocol.Content
 
disconnect() - Method in class org.apache.nutch.protocol.ftp.Client
Closes the connection to the FTP server and restores connection parameters to the default values.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Get a float value from Fetcher.SCORE_KEY, divide it by the number of outlinks and apply.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in interface org.apache.nutch.scoring.ScoringFilter
Distribute score value from the current page to all its outlinked pages.
distributeScoreToOutlinks(Text, ParseData, Collection<Map.Entry<Text, CrawlDatum>>, CrawlDatum, int) - Method in class org.apache.nutch.scoring.ScoringFilters
 
DmozParser - Class in org.apache.nutch.tools
Utility that converts DMOZ RDF into a flat file of URLs to be injected.
DmozParser() - Constructor for class org.apache.nutch.tools.DmozParser
 
doFilter(ServletRequest, ServletResponse, FilterChain) - Method in class org.apache.nutch.tools.proxy.LogDebugHandler
 
DomainStatistics - Class in org.apache.nutch.util.domain
Extracts some very basic statistics about domains from the crawldb
DomainStatistics() - Constructor for class org.apache.nutch.util.domain.DomainStatistics
 
DomainStatistics.DomainStatisticsCombiner - Class in org.apache.nutch.util.domain
 
DomainStatistics.DomainStatisticsCombiner() - Constructor for class org.apache.nutch.util.domain.DomainStatistics.DomainStatisticsCombiner
 
DomainStatistics.MyCounter - Enum in org.apache.nutch.util.domain
 
DomainSuffix - Class in org.apache.nutch.util.domain
This class represents the last part of the host name, which is operated by authoritives, not individuals.
DomainSuffix(String, DomainSuffix.Status, float) - Constructor for class org.apache.nutch.util.domain.DomainSuffix
 
DomainSuffix(String) - Constructor for class org.apache.nutch.util.domain.DomainSuffix
 
DomainSuffix.Status - Enum in org.apache.nutch.util.domain
Enumeration of the status of the tld.
DomainSuffixes - Class in org.apache.nutch.util.domain
Storage class for DomainSuffix objects Note: this class is singleton
DomUtil - Class in org.apache.nutch.util
 
DomUtil() - Constructor for class org.apache.nutch.util.DomUtil
 
DublinCore - Interface in org.apache.nutch.metadata
A collection of Dublin Core metadata names.
DummySSLProtocolSocketFactory - Class in org.apache.nutch.protocol.httpclient
 
DummySSLProtocolSocketFactory() - Constructor for class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
Constructor for DummySSLProtocolSocketFactory.
DummyX509TrustManager - Class in org.apache.nutch.protocol.httpclient
 
DummyX509TrustManager(KeyStore) - Constructor for class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
Constructor for DummyX509TrustManager.
dump(Path, Path) - Method in class org.apache.nutch.segment.SegmentReader
 
DUMP_DIR - Static variable in class org.apache.nutch.scoring.webgraph.LinkDumper
 
dumpLinks(Path) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper
Runs the inverter and merger jobs of the LinkDumper tool to create the url to inlink node database.
dumpNodes(Path, NodeDumper.DumpType, long, Path) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper
Runs the process to dump the top urls out to a text file.
dumpUrl(Path, String) - Method in class org.apache.nutch.scoring.webgraph.LoopReader
Prints loopset for a single url.
dumpUrl(Path, String) - Method in class org.apache.nutch.scoring.webgraph.NodeReader
Prints the content of the Node represented by the url to system out.

E

elapsedTime(long, long) - Static method in class org.apache.nutch.util.TimingUtil
Calculate the elapsed time between two times specified in milliseconds.
EmptyRobotRules - Class in org.apache.nutch.protocol
 
EmptyRobotRules() - Constructor for class org.apache.nutch.protocol.EmptyRobotRules
 
EncodingDetector - Class in org.apache.nutch.util
A simple class for detecting character encodings.
EncodingDetector(Configuration) - Constructor for class org.apache.nutch.util.EncodingDetector
 
end - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
equals(Object) - Method in class org.apache.nutch.crawl.CrawlDatum
 
equals(Object) - Method in class org.apache.nutch.crawl.Inlink
 
equals(Object) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
equals(Object) - Method in class org.apache.nutch.fetcher.FetcherOutput
 
equals(Object) - Method in class org.apache.nutch.metadata.Metadata
 
equals(Object) - Method in class org.apache.nutch.parse.Outlink
 
equals(Object) - Method in class org.apache.nutch.parse.ParseData
 
equals(Object) - Method in class org.apache.nutch.parse.ParseStatus
 
equals(Object) - Method in class org.apache.nutch.parse.ParseText
 
equals(Object) - Method in class org.apache.nutch.protocol.Content
 
equals(Object) - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
 
equals(Object) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
evaluate() - Method in class org.apache.nutch.util.CommandRunner
 
EXCEPTION - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Unspecified exception occured.
exec() - Method in class org.apache.nutch.util.CommandRunner
 
Extension - Class in org.apache.nutch.plugin
An Extension is a kind of listener descriptor that will be installed on a concrete ExtensionPoint that acts as kind of Publisher.
Extension(PluginDescriptor, String, String, String, Configuration, PluginRepository) - Constructor for class org.apache.nutch.plugin.Extension
 
ExtensionPoint - Class in org.apache.nutch.plugin
The ExtensionPoint provide meta information of a extension point.
ExtensionPoint(String, String, String) - Constructor for class org.apache.nutch.plugin.ExtensionPoint
Constructor
ExtParser - Class in org.apache.nutch.parse.ext
A wrapper that invokes external command to do real parsing job.
ExtParser() - Constructor for class org.apache.nutch.parse.ext.ExtParser
 
extractText(InputStream, String, List) - Method in class org.apache.nutch.parse.zip.ZipTextExtractor
 

F

FAILED - Static variable in class org.apache.nutch.parse.ParseStatus
General failure.
FAILED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Content was not retrieved.
FAILED_EXCEPTION - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing failed.
FAILED_INVALID_FORMAT - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing failed.
FAILED_MISSING_CONTENT - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing failed.
FAILED_MISSING_PARTS - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing failed.
FAILED_TRUNCATED - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing failed.
FakeHandler - Class in org.apache.nutch.tools.proxy
 
FakeHandler() - Constructor for class org.apache.nutch.tools.proxy.FakeHandler
 
Feed - Interface in org.apache.nutch.metadata
A collection of Feed property names extracted by the ROME library.
FEED - Static variable in interface org.apache.nutch.metadata.Feed
 
FEED_AUTHOR - Static variable in interface org.apache.nutch.metadata.Feed
 
FEED_PUBLISHED - Static variable in interface org.apache.nutch.metadata.Feed
 
FEED_TAGS - Static variable in interface org.apache.nutch.metadata.Feed
 
FEED_UPDATED - Static variable in interface org.apache.nutch.metadata.Feed
 
fetch(Path, int, boolean) - Method in class org.apache.nutch.fetcher.Fetcher
 
fetch(Path, int) - Method in class org.apache.nutch.fetcher.OldFetcher
 
FETCH_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
 
FETCH_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
FETCH_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
fetched - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
Fetcher - Class in org.apache.nutch.fetcher
A queue-based fetcher.
Fetcher() - Constructor for class org.apache.nutch.fetcher.Fetcher
 
Fetcher(Configuration) - Constructor for class org.apache.nutch.fetcher.Fetcher
 
Fetcher.InputFormat - Class in org.apache.nutch.fetcher
 
Fetcher.InputFormat() - Constructor for class org.apache.nutch.fetcher.Fetcher.InputFormat
 
FetcherOutput - Class in org.apache.nutch.fetcher
 
FetcherOutput() - Constructor for class org.apache.nutch.fetcher.FetcherOutput
 
FetcherOutput(CrawlDatum, Content, ParseImpl) - Constructor for class org.apache.nutch.fetcher.FetcherOutput
 
FetcherOutputFormat - Class in org.apache.nutch.fetcher
Splits FetcherOutput entries into multiple map files.
FetcherOutputFormat() - Constructor for class org.apache.nutch.fetcher.FetcherOutputFormat
 
fetchErrors - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
FetchSchedule - Interface in org.apache.nutch.crawl
This interface defines the contract for implementations that manipulate fetch times and re-fetch intervals.
FetchScheduleFactory - Class in org.apache.nutch.crawl
Creates and caches a FetchSchedule implementation.
FIELD - Static variable in class org.creativecommons.nutch.CCIndexingFilter
The name of the document field we use.
File - Class in org.apache.nutch.protocol.file
File.java deals with file: scheme.
File() - Constructor for class org.apache.nutch.protocol.file.File
 
FileError - Exception in org.apache.nutch.protocol.file
Thrown for File error codes.
FileError(int) - Constructor for exception org.apache.nutch.protocol.file.FileError
 
FileException - Exception in org.apache.nutch.protocol.file
 
FileException() - Constructor for exception org.apache.nutch.protocol.file.FileException
 
FileException(String) - Constructor for exception org.apache.nutch.protocol.file.FileException
 
FileException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.file.FileException
 
FileException(Throwable) - Constructor for exception org.apache.nutch.protocol.file.FileException
 
fileLen - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
FileResponse - Class in org.apache.nutch.protocol.file
FileResponse.java mimics file replies as http response.
FileResponse(URL, CrawlDatum, File, Configuration) - Constructor for class org.apache.nutch.protocol.file.FileResponse
 
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
Scan the HTML document looking at possible indications of content language
1.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
 
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
 
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in interface org.apache.nutch.indexer.IndexingFilter
Adds fields or otherwise modifies the document that will be indexed for a parse.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.IndexingFilters
Run all defined filters.
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
 
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
 
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.microformats.reltag.RelTagParser
Scan the HTML document looking at possible rel-tags
filter(String) - Method in interface org.apache.nutch.net.URLFilter
 
filter(String) - Method in class org.apache.nutch.net.URLFilters
Run all defined filters.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in interface org.apache.nutch.parse.HtmlParseFilter
Adds metadata or otherwise modifies a parse of HTML content, given the DOM tree of a page.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.HtmlParseFilters
Run all defined filters.
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.apache.nutch.parse.js.JSParseFilter
 
filter() - Method in class org.apache.nutch.parse.ParseResult
Remove all results where status is not successful (as determined by ParseStatus.isSuccess()).
filter(WritableComparable, CrawlDatum, CrawlDatum, CrawlDatum, Content, ParseData, ParseText, Collection<CrawlDatum>) - Method in interface org.apache.nutch.segment.SegmentMergeFilter
The filtering method which gets all information being merged for a given key (URL).
filter(WritableComparable, CrawlDatum, CrawlDatum, CrawlDatum, Content, ParseData, ParseText, Collection<CrawlDatum>) - Method in class org.apache.nutch.segment.SegmentMergeFilters
Iterates over all SegmentMergeFilter extensions and if any of them returns false, it will return false as well.
filter(String) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
 
filter(String) - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
filter(NutchDocument, Parse, Text, CrawlDatum, Inlinks) - Method in class org.creativecommons.nutch.CCIndexingFilter
 
filter(Content, ParseResult, HTMLMetaTags, DocumentFragment) - Method in class org.creativecommons.nutch.CCParseFilter
Adds metadata or otherwise modifies a parse of an HTML document, given the DOM tree of a page.
finalize() - Method in class org.apache.nutch.plugin.Plugin
 
finalize() - Method in class org.apache.nutch.plugin.PluginRepository
 
finalize() - Method in class org.apache.nutch.protocol.ftp.Ftp
 
findAuthentication(Metadata) - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
 
findLoops(Path) - Method in class org.apache.nutch.scoring.webgraph.Loops
Runs the various loop jobs.
forceRefetch(Text, CrawlDatum, boolean) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
This method resets fetchTime, fetchInterval, modifiedTime, retriesSinceFetch and page signature, so that it forces refetching.
forceRefetch(Text, CrawlDatum, boolean) - Method in interface org.apache.nutch.crawl.FetchSchedule
This method resets fetchTime, fetchInterval, modifiedTime and page signature, so that it forces refetching.
FORMAT - Static variable in interface org.apache.nutch.metadata.DublinCore
Typically, Format may include the media-type or dimensions of the resource.
format - Static variable in class org.apache.nutch.net.protocols.HttpDateFormat
 
forName(String) - Method in class org.apache.nutch.util.MimeUtil
A facade interface to Tika's underlying MimeTypes.forName(String) method.
FreeGenerator - Class in org.apache.nutch.tools
This tool generates fetchlists (segments to be fetched) from plain text files containing one URL per line.
FreeGenerator() - Constructor for class org.apache.nutch.tools.FreeGenerator
 
FreeGenerator.FG - Class in org.apache.nutch.tools
 
FreeGenerator.FG() - Constructor for class org.apache.nutch.tools.FreeGenerator.FG
 
fromHexString(String) - Static method in class org.apache.nutch.util.StringUtil
Convert a String containing consecutive (no inside whitespace) hexadecimal digits into a corresponding byte array.
FSUtils - Class in org.apache.nutch.util
Utility methods for common filesystem operations.
FSUtils() - Constructor for class org.apache.nutch.util.FSUtils
 
Ftp - Class in org.apache.nutch.protocol.ftp
Ftp.java deals with ftp: scheme.
Ftp() - Constructor for class org.apache.nutch.protocol.ftp.Ftp
 
FtpError - Exception in org.apache.nutch.protocol.ftp
Thrown for Ftp error codes.
FtpError(int) - Constructor for exception org.apache.nutch.protocol.ftp.FtpError
 
FtpException - Exception in org.apache.nutch.protocol.ftp
Superclass for important exceptions thrown during FTP talk, that must be handled with care.
FtpException() - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
 
FtpException(String) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
 
FtpException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
 
FtpException(Throwable) - Constructor for exception org.apache.nutch.protocol.ftp.FtpException
 
FtpExceptionBadSystResponse - Exception in org.apache.nutch.protocol.ftp
Exception indicating bad reply of SYST command.
FtpExceptionCanNotHaveDataConnection - Exception in org.apache.nutch.protocol.ftp
Exception indicating failure of opening data connection.
FtpExceptionControlClosedByForcedDataClose - Exception in org.apache.nutch.protocol.ftp
Exception indicating control channel is closed by server end, due to forced closure of data channel at client (our) end.
FtpExceptionUnknownForcedDataClose - Exception in org.apache.nutch.protocol.ftp
Exception indicating unrecognizable reply from server after forced closure of data channel by client (our) side.
FtpResponse - Class in org.apache.nutch.protocol.ftp
FtpResponse.java mimics ftp replies as http response.
FtpResponse(URL, CrawlDatum, Ftp, Configuration) - Constructor for class org.apache.nutch.protocol.ftp.FtpResponse
 

G

generate(Path, Path, int, long, long) - Method in class org.apache.nutch.crawl.Generator
 
generate(Path, Path, int, long, long, boolean, boolean) - Method in class org.apache.nutch.crawl.Generator
old signature used for compatibility - does not specify whether or not to normalise and set the number of segments to 1
generate(Path, Path, int, long, long, boolean, boolean, boolean, int) - Method in class org.apache.nutch.crawl.Generator
Generate fetchlists in one or more segments.
GENERATE_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
 
GENERATE_MAX_PER_HOST - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATE_MAX_PER_HOST_BY_IP - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATE_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
GENERATE_UPDATE_CRAWLDB - Static variable in class org.apache.nutch.crawl.Generator
 
generated - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
generateFileNameForKeyValue(FloatWritable, Generator.SelectorEntry, String) - Method in class org.apache.nutch.crawl.Generator.GeneratorOutputFormat
 
generateSegmentName() - Static method in class org.apache.nutch.crawl.Generator
 
generateSegmentName() - Static method in class org.apache.nutch.tools.arc.ArcSegmentCreator
Generates a random name for the segments.
Generator - Class in org.apache.nutch.crawl
Generates a subset of a crawl db to fetch.
Generator() - Constructor for class org.apache.nutch.crawl.Generator
 
Generator(Configuration) - Constructor for class org.apache.nutch.crawl.Generator
 
Generator.CrawlDbUpdater - Class in org.apache.nutch.crawl
Update the CrawlDB so that the next generate won't include the same URLs.
Generator.CrawlDbUpdater() - Constructor for class org.apache.nutch.crawl.Generator.CrawlDbUpdater
 
Generator.DecreasingFloatComparator - Class in org.apache.nutch.crawl
 
Generator.DecreasingFloatComparator() - Constructor for class org.apache.nutch.crawl.Generator.DecreasingFloatComparator
 
Generator.GeneratorOutputFormat - Class in org.apache.nutch.crawl
 
Generator.GeneratorOutputFormat() - Constructor for class org.apache.nutch.crawl.Generator.GeneratorOutputFormat
 
Generator.HashComparator - Class in org.apache.nutch.crawl
Sort fetch lists by hash of URL.
Generator.HashComparator() - Constructor for class org.apache.nutch.crawl.Generator.HashComparator
 
Generator.PartitionReducer - Class in org.apache.nutch.crawl
 
Generator.PartitionReducer() - Constructor for class org.apache.nutch.crawl.Generator.PartitionReducer
 
Generator.Selector - Class in org.apache.nutch.crawl
Selects entries due for fetch.
Generator.Selector() - Constructor for class org.apache.nutch.crawl.Generator.Selector
 
Generator.SelectorEntry - Class in org.apache.nutch.crawl
 
Generator.SelectorEntry() - Constructor for class org.apache.nutch.crawl.Generator.SelectorEntry
 
Generator.SelectorInverseMapper - Class in org.apache.nutch.crawl
 
Generator.SelectorInverseMapper() - Constructor for class org.apache.nutch.crawl.Generator.SelectorInverseMapper
 
GENERATOR_COUNT_MODE - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_COUNT_VALUE_DOMAIN - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_COUNT_VALUE_HOST - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_CUR_TIME - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_DELAY - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_FILTER - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_MAX_COUNT - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_MAX_NUM_SEGMENTS - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_MIN_SCORE - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_NORMALISE - Static variable in class org.apache.nutch.crawl.Generator
 
GENERATOR_TOP_N - Static variable in class org.apache.nutch.crawl.Generator
 
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Use CrawlDatum.getScore().
generatorSortValue(Text, CrawlDatum, float) - Method in interface org.apache.nutch.scoring.ScoringFilter
This method prepares a sort value for the purpose of sorting and selecting top N scoring pages during fetchlist generation.
generatorSortValue(Text, CrawlDatum, float) - Method in class org.apache.nutch.scoring.ScoringFilters
Calculate a sort value for Generate.
GenericWritableConfigurable - Class in org.apache.nutch.util
A generic Writable wrapper that can inject Configuration to Configurables
GenericWritableConfigurable() - Constructor for class org.apache.nutch.util.GenericWritableConfigurable
 
get(String, String, Configuration) - Method in class org.apache.nutch.crawl.CrawlDbReader
 
get(Writable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
get(String) - Method in class org.apache.nutch.metadata.Metadata
Get the value associated to a metadata name.
get(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
 
get(String) - Method in class org.apache.nutch.parse.ParseResult
Retrieve a single parse output.
get(Text) - Method in class org.apache.nutch.parse.ParseResult
Retrieve a single parse output.
get(Configuration) - Static method in class org.apache.nutch.plugin.PluginRepository
 
get(FileSplit) - Static method in class org.apache.nutch.segment.SegmentPart
Create SegmentPart from a FileSplit.
get(String) - Static method in class org.apache.nutch.segment.SegmentPart
Create SegmentPart from a full path of a location inside any segment part.
get(Path, Text, Writer, Map<String, List<Writable>>) - Method in class org.apache.nutch.segment.SegmentReader
 
get(String) - Method in class org.apache.nutch.util.domain.DomainSuffixes
Return the DomainSuffix object for the extension, if extension is a top level domain returned object will be an instance of TopLevelDomain
get(Configuration) - Static method in class org.apache.nutch.util.ObjectCache
 
getAcceptedIssuers() - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
 
getAcceptLanguage() - Method in class org.apache.nutch.protocol.http.api.HttpBase
Value of "Accept-Language" request header sent by Nutch.
getAnchor() - Method in class org.apache.nutch.crawl.Inlink
 
getAnchor() - Method in class org.apache.nutch.parse.Outlink
 
getAnchor() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
getAnchors() - Method in class org.apache.nutch.crawl.Inlinks
Return the set of anchor texts.
getAnchors(Text) - Method in class org.apache.nutch.crawl.LinkDbReader
 
getArgs() - Method in class org.apache.nutch.parse.ParseStatus
 
getArgs() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
getAttribute(String) - Method in class org.apache.nutch.plugin.Extension
Returns a attribute value, that is setuped in the manifest file and is definied by the extension point xml schema.
getAuthentication(String, Configuration) - Static method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
This method is responsible for providing Basic authentication information.
getBaseHref() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getBaseUrl() - Method in class org.apache.nutch.protocol.Content
The base url for relative links contained in the content.
getBasicPattern() - Static method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
Provides a pattern which can be used by an outside resource to determine if this class can provide credentials based on simple header information.
getBoost() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
getBoost() - Method in class org.apache.nutch.util.domain.DomainSuffix
 
getClassLoader() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a cached classloader for a plugin.
getClazz() - Method in class org.apache.nutch.plugin.Extension
Returns the full class name of the extension point implementation
getCode() - Method in interface org.apache.nutch.net.protocols.Response
Returns the response code.
getCode(int) - Method in exception org.apache.nutch.protocol.file.FileError
 
getCode() - Method in class org.apache.nutch.protocol.file.FileResponse
Returns the response code.
getCode(int) - Method in exception org.apache.nutch.protocol.ftp.FtpError
 
getCode() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
Returns the response code.
getCode() - Method in class org.apache.nutch.protocol.http.HttpResponse
 
getCode() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
 
getCode() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
getCommand() - Method in class org.apache.nutch.util.CommandRunner
 
getConf() - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
 
getConf() - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
 
getConf() - Method in class org.apache.nutch.crawl.Signature
 
getConf() - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
 
getConf() - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
 
getConf() - Method in class org.apache.nutch.indexer.solr.SolrClean
 
getConf() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
getConf() - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
 
getConf() - Method in class org.apache.nutch.microformats.reltag.RelTagParser
 
getConf() - Method in class org.apache.nutch.parse.ext.ExtParser
 
getConf() - Method in class org.apache.nutch.parse.js.JSParseFilter
 
getConf() - Method in class org.apache.nutch.parse.swf.SWFParser
 
getConf() - Method in class org.apache.nutch.parse.tika.TikaParser
 
getConf() - Method in class org.apache.nutch.parse.zip.ZipParser
 
getConf() - Method in class org.apache.nutch.protocol.file.File
 
getConf() - Method in class org.apache.nutch.protocol.ftp.Ftp
 
getConf() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getConf() - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
getConf() - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
 
getConf() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
 
getConf() - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
 
getConf() - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
 
getConf() - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
getConf() - Method in class org.apache.nutch.util.domain.DomainStatistics
 
getConf() - Method in class org.apache.nutch.util.GenericWritableConfigurable
 
getConf() - Method in class org.creativecommons.nutch.CCIndexingFilter
 
getConf() - Method in class org.creativecommons.nutch.CCParseFilter
 
getContent() - Method in class org.apache.nutch.fetcher.FetcherOutput
 
getContent() - Method in interface org.apache.nutch.net.protocols.Response
Returns the full content of the response.
getContent() - Method in class org.apache.nutch.protocol.Content
The binary content retrieved.
getContent() - Method in class org.apache.nutch.protocol.file.FileResponse
 
getContent() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
 
getContent() - Method in class org.apache.nutch.protocol.http.HttpResponse
 
getContent() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
 
getContent() - Method in class org.apache.nutch.protocol.ProtocolOutput
 
getContentMeta() - Method in class org.apache.nutch.parse.ParseData
The original Metadata retrieved from content
getContentType() - Method in exception org.apache.nutch.parse.ParserNotFound
 
getContentType() - Method in class org.apache.nutch.protocol.Content
The media type of the retrieved content.
getCopyMap() - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
getCountryName() - Method in class org.apache.nutch.util.domain.TopLevelDomain
Returns the country name if TLD is Country Code TLD
getCrawlDatum() - Method in class org.apache.nutch.fetcher.FetcherOutput
 
getCrawlDelay() - Method in class org.apache.nutch.protocol.EmptyRobotRules
 
getCrawlDelay(HttpBase, URL) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
getCrawlDelay() - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Get Crawl-Delay, in milliseconds.
getCrawlDelay() - Method in interface org.apache.nutch.protocol.RobotRules
Get Crawl-Delay, in milliseconds.
getCredentials() - Method in interface org.apache.nutch.protocol.httpclient.HttpAuthentication
Gets the credentials generated by the HttpAuthentication object.
getCredentials() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
Gets the Basic credentials generated by this HttpBasicAuthentication object
getData() - Method in interface org.apache.nutch.parse.Parse
Other data extracted from the page.
getData() - Method in class org.apache.nutch.parse.ParseImpl
 
getDebugStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getDefaultConfig() - Static method in class org.apache.nutch.parse.tika.TikaConfig
Provides a default configuration (TikaConfig).
getDefaultConfig(Parser) - Static method in class org.apache.nutch.parse.tika.TikaConfig
Deprecated. This method will be removed in Apache Tika 1.0
getDependencies() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a array of plugin ids.
getDescriptor() - Method in class org.apache.nutch.plugin.Extension
return the plugin descriptor.
getDescriptor() - Method in class org.apache.nutch.plugin.Plugin
Returns the plugin descriptor
getDocBegin() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
getDocumentMeta() - Method in class org.apache.nutch.indexer.NutchDocument
 
getDom(InputStream) - Static method in class org.apache.nutch.util.DomUtil
Returns parsed dom tree or null if any error
getDomain() - Method in class org.apache.nutch.util.domain.DomainSuffix
 
getDomainName(URL) - Static method in class org.apache.nutch.util.URLUtil
Returns the domain name of the url.
getDomainName(String) - Static method in class org.apache.nutch.util.URLUtil
Returns the domain name of the url.
getDomainSuffix(URL) - Static method in class org.apache.nutch.util.URLUtil
Returns the DomainSuffix corresponding to the last public part of the hostname
getDomainSuffix(String) - Static method in class org.apache.nutch.util.URLUtil
Returns the DomainSuffix corresponding to the last public part of the hostname
getEmptyParse(Configuration) - Method in class org.apache.nutch.parse.ParseStatus
A convenience method.
getEmptyParseResult(String, Configuration) - Method in class org.apache.nutch.parse.ParseStatus
A convenience method.
getErrorStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getExitValue() - Method in class org.apache.nutch.util.CommandRunner
 
getExpireTime() - Method in class org.apache.nutch.protocol.EmptyRobotRules
 
getExpireTime() - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Get expire time
getExpireTime() - Method in interface org.apache.nutch.protocol.RobotRules
Get expire time
getExportedLibUrls() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a array exported librareis as URLs
getExtensionInstance() - Method in class org.apache.nutch.plugin.Extension
Return an instance of the extension implementatio.
getExtensionPoint(String) - Method in class org.apache.nutch.plugin.PluginRepository
Returns a extension point indentified by a extension point id.
getExtensions(String) - Method in class org.apache.nutch.parse.ParserFactory
Finds the best-suited parse plugin for a given contentType.
getExtensions() - Method in class org.apache.nutch.plugin.ExtensionPoint
Returns a array of extensions that lsiten to this extension point
getExtensions() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns an array of extensions.
getExtenstionPoints() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a array of extension points.
getFatalStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getFetchInterval() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getFetchSchedule(Configuration) - Static method in class org.apache.nutch.crawl.FetchScheduleFactory
Return the FetchSchedule implementation.
getFetchTime() - Method in class org.apache.nutch.crawl.CrawlDatum
Returns either the time of the last fetch, or the next fetch time, depending on whether Fetcher or CrawlDbReducer set the time.
getField(String) - Method in class org.apache.nutch.indexer.NutchDocument
 
getFieldNames() - Method in class org.apache.nutch.indexer.NutchDocument
 
getFieldValue(String) - Method in class org.apache.nutch.indexer.NutchDocument
 
getFromUrl() - Method in class org.apache.nutch.crawl.Inlink
 
getGeneralTags() - Method in class org.apache.nutch.parse.HTMLMetaTags
Returns all collected values of the general meta tags.
getHeader(String) - Method in interface org.apache.nutch.net.protocols.Response
Returns the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.file.FileResponse
Returns the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.ftp.FtpResponse
Returns the value of a named header.
getHeader(String) - Method in class org.apache.nutch.protocol.http.HttpResponse
 
getHeader(String) - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
 
getHeaders() - Method in interface org.apache.nutch.net.protocols.Response
Returns all the headers.
getHeaders() - Method in class org.apache.nutch.protocol.http.HttpResponse
 
getHeaders() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
 
getHost(String) - Static method in class org.apache.nutch.util.URLUtil
Returns the lowercased hostname for the url or null if the url is not well formed.
getHostSegments(URL) - Static method in class org.apache.nutch.util.URLUtil
Partitions of the hostname of the url by "."
getHostSegments(String) - Static method in class org.apache.nutch.util.URLUtil
Partitions of the hostname of the url by "."
getHttpEquivTags() - Method in class org.apache.nutch.parse.HTMLMetaTags
Returns all collected values of the "http-equiv" meta tags.
getId() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
getId() - Method in class org.apache.nutch.plugin.Extension
Return the unique id of the extension.
getId() - Method in class org.apache.nutch.plugin.ExtensionPoint
Returns the unique id of the extension point.
getInfoStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getInlinks(Text) - Method in class org.apache.nutch.crawl.LinkDbReader
 
getInlinkScore() - Method in class org.apache.nutch.scoring.webgraph.Node
 
getInstance(Configuration) - Static method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
getInstance() - Static method in class org.apache.nutch.util.domain.DomainSuffixes
Singleton instance, lazy instantination
getKeyMap() - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
getLastModified() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
getLength() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
getLinks() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
getLinkType() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
getLocations() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
getLookingFor() - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
getLoopSet() - Method in class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
getMajorCode() - Method in class org.apache.nutch.parse.ParseStatus
 
getMaxContent() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getMessage() - Method in class org.apache.nutch.parse.ParseStatus
A convenience method.
getMessage() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
getMeta(String) - Method in class org.apache.nutch.metadata.MetaWrapper
Get metadata.
getMeta(String) - Method in class org.apache.nutch.parse.ParseData
Get a metadata single value.
getMetaData() - Method in class org.apache.nutch.crawl.CrawlDatum
returns a MapWritable if it was set or read in @see readFields(DataInput), returns empty map in case CrawlDatum was freshly created (lazily instantiated).
getMetadata() - Method in class org.apache.nutch.metadata.MetaWrapper
Get all metadata.
getMetadata() - Method in class org.apache.nutch.protocol.Content
Other protocol-specific data.
getMetadata() - Method in class org.apache.nutch.scoring.webgraph.Node
 
getMetaValues(String) - Method in class org.apache.nutch.metadata.MetaWrapper
Get multiple metadata.
getMimeRepository() - Method in class org.apache.nutch.parse.tika.TikaConfig
 
getMimeType(String) - Method in class org.apache.nutch.util.MimeUtil
Facade interface to Tika's underlying MimeTypes.getMimeType(String) method.
getMimeType(File) - Method in class org.apache.nutch.util.MimeUtil
Facade interface to Tika's underlying MimeTypes.getMimeType(File) method.
getMinorCode() - Method in class org.apache.nutch.parse.ParseStatus
 
getModifiedTime() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getName() - Method in class org.apache.nutch.analysis.lang.NGramProfile
 
getName() - Method in class org.apache.nutch.plugin.ExtensionPoint
Returns the name of the extension point.
getName() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns the name of the plugin.
getName() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
getNoCache() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getNode() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
getNoFollow() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getNoIndex() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getNormalizedName(String) - Static method in class org.apache.nutch.metadata.SpellCheckedMetadata
Get the normalized name of metadata attribute name.
getNotExportedLibUrls() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a array of libraries as URLs that are not exported by the plugin.
getNumDocs() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
getNumInlinks() - Method in class org.apache.nutch.scoring.webgraph.Node
 
getNumOutlinks() - Method in class org.apache.nutch.scoring.webgraph.Node
 
getNutchIndexWriters(Configuration) - Static method in class org.apache.nutch.indexer.NutchIndexWriterFactory
 
getObject(String) - Method in class org.apache.nutch.util.ObjectCache
 
getOutlinks(String, Configuration) - Static method in class org.apache.nutch.parse.OutlinkExtractor
Extracts Outlink from given plain text.
getOutlinks(String, String, Configuration) - Static method in class org.apache.nutch.parse.OutlinkExtractor
Extracts Outlink from given plain text and adds anchor to the extracted Outlinks
getOutlinks() - Method in class org.apache.nutch.parse.ParseData
The outlinks of the page.
getOutlinkScore() - Method in class org.apache.nutch.scoring.webgraph.Node
 
getOutlinkUrl() - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
getPage(String) - Static method in class org.apache.nutch.util.URLUtil
Returns the page for the url.
getParse() - Method in class org.apache.nutch.fetcher.FetcherOutput
 
getParse(Content) - Method in class org.apache.nutch.parse.ext.ExtParser
 
getParse(Content) - Method in class org.apache.nutch.parse.js.JSParseFilter
 
getParse(Content) - Method in interface org.apache.nutch.parse.Parser
This method parses the given content and returns a map of <key, parse> pairs.
getParse(Content) - Method in class org.apache.nutch.parse.swf.SWFParser
 
getParse(Content) - Method in class org.apache.nutch.parse.tika.TikaParser
 
getParse(Content) - Method in class org.apache.nutch.parse.zip.ZipParser
 
getParseMeta() - Method in class org.apache.nutch.parse.ParseData
Other content properties.
getParser(String) - Method in class org.apache.nutch.parse.tika.TikaConfig
Returns the parser instance configured for the given MIME type.
getParserById(String) - Method in class org.apache.nutch.parse.ParserFactory
Function returns a Parser instance with the specified extId, representing its extension ID.
getParsers(String, String) - Method in class org.apache.nutch.parse.ParserFactory
Function returns an array of Parsers for a given content type.
getParsers() - Method in class org.apache.nutch.parse.tika.TikaConfig
 
getPartition(FloatWritable, Writable, int) - Method in class org.apache.nutch.crawl.Generator.Selector
Partition by host / domain or IP.
getPartition(Text, Writable, int) - Method in class org.apache.nutch.crawl.URLPartitioner
Hash by domain name.
getPassAllFilter() - Static method in class org.apache.nutch.util.HadoopFSUtil
Returns PathFilter that passes all paths through.
getPassDirectoriesFilter(FileSystem) - Static method in class org.apache.nutch.util.HadoopFSUtil
Returns PathFilter that passes directories through.
getPaths(FileStatus[]) - Static method in class org.apache.nutch.util.HadoopFSUtil
Turns an array of FileStatus into an array of Paths.
getPluginClass() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns the fully qualified name of the class which implements the abstarct Plugin class.
getPluginDescriptor(String) - Method in class org.apache.nutch.plugin.PluginRepository
Returns the descriptor of one plugin identified by a plugin id.
getPluginDescriptors() - Method in class org.apache.nutch.plugin.PluginRepository
Returns all registed plugin descriptors.
getPluginFolder(String) - Method in class org.apache.nutch.plugin.PluginManifestParser
Return the named plugin folder.
getPluginId() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns the unique identifier of the plug-in or null.
getPluginInstance(PluginDescriptor) - Method in class org.apache.nutch.plugin.PluginRepository
Returns a instance of a plugin.
getPluginPath() - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns the directory path of the plugin.
getPos() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Returns the current position in the file.
getProgress() - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Returns the percentage of progress in processing the file.
getProtocol(String) - Method in class org.apache.nutch.protocol.ProtocolFactory
Returns the appropriate Protocol implementation for a url.
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.file.File
 
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.ftp.Ftp
 
getProtocolOutput(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getProtocolOutput(Text, CrawlDatum) - Method in interface org.apache.nutch.protocol.Protocol
Returns the Content for a fetchlist entry.
getProviderName() - Method in class org.apache.nutch.plugin.PluginDescriptor
 
getProxyHost() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getProxyPort() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getRealm() - Method in interface org.apache.nutch.protocol.httpclient.HttpAuthentication
Gets the realm used by the HttpAuthentication object during creation.
getRealm() - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
Gets the realm attribute of the HttpBasicAuthentication object.
getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputFormat
 
getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.nutch.segment.ContentAsTextInputFormat
 
getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.nutch.segment.SegmentMerger.ObjectInputFormat
 
getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.nutch.tools.arc.ArcInputFormat
Returns the RecordReader for reading the arc file.
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat
 
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.fetcher.FetcherOutputFormat
 
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.indexer.IndexerOutputFormat
 
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.parse.ParseOutputFormat
 
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.segment.SegmentMerger.SegmentOutputFormat
 
getRecordWriter(FileSystem, JobConf, String, Progressable) - Method in class org.apache.nutch.segment.SegmentReader.TextOutputFormat
 
getRefresh() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getRefreshHref() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getRefreshTime() - Method in class org.apache.nutch.parse.HTMLMetaTags
A convenience method.
getResourceString(String, Locale) - Method in class org.apache.nutch.plugin.PluginDescriptor
Returns a I18N'd resource string.
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.http.Http
 
getResponse(URL, CrawlDatum, boolean) - Method in class org.apache.nutch.protocol.httpclient.Http
Fetches the url with a configured HTTP client and gets the response.
getRetriesSinceFetch() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getRobotRules(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.file.File
 
getRobotRules(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.ftp.Ftp
 
getRobotRules(Text, CrawlDatum) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getRobotRules(Text, CrawlDatum) - Method in interface org.apache.nutch.protocol.Protocol
Retrieve robot rules applicable for this url.
getRobotRulesSet(HttpBase, Text) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Returns the name of the file of rules to use for a particular implementation.
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
Rules specified as a config property will override rules specified as a config file.
getRulesReader(Configuration) - Method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
Rules specified as a config property will override rules specified as a config file.
getRuns() - Method in class org.apache.nutch.tools.Benchmark.BenchmarkResults
 
getSchema() - Method in class org.apache.nutch.plugin.ExtensionPoint
Returns a path to the xml schema of a extension point.
getScore() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getScore() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
getSignature() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getSignature(Configuration) - Static method in class org.apache.nutch.crawl.SignatureFactory
Return the default Signature implementation.
getSimilarity(NGramProfile) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Calculate a score how well NGramProfiles match each other
getSorted() - Method in class org.apache.nutch.analysis.lang.NGramProfile
Return a sorted list of ngrams (sort done by 1.
getSplits(JobConf, int) - Method in class org.apache.nutch.fetcher.Fetcher.InputFormat
Don't split inputs, to keep things polite.
getSplits(JobConf, int) - Method in class org.apache.nutch.fetcher.OldFetcher.InputFormat
Don't split inputs, to keep things polite.
getSplits(JobConf, int) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputFormat
Return each index as a split.
getStages() - Method in class org.apache.nutch.tools.Benchmark.BenchmarkResults
 
getStats(Path, SegmentReader.SegmentReaderStats) - Method in class org.apache.nutch.segment.SegmentReader
 
getStatus() - Method in class org.apache.nutch.crawl.CrawlDatum
 
getStatus() - Method in class org.apache.nutch.parse.ParseData
The status of parsing the page.
getStatus() - Method in class org.apache.nutch.protocol.ProtocolOutput
 
getStatus() - Method in class org.apache.nutch.util.domain.DomainSuffix
 
getStatusName(byte) - Static method in class org.apache.nutch.crawl.CrawlDatum
 
getSystemName() - Method in class org.apache.nutch.protocol.ftp.Client
Fetches the system type name from the server and returns the string.
getTargetPoint() - Method in class org.apache.nutch.plugin.Extension
Returns the Id of the extension point, that is implemented by this extension.
getText() - Method in interface org.apache.nutch.parse.Parse
The textual content of the page.
getText() - Method in class org.apache.nutch.parse.ParseImpl
 
getText() - Method in class org.apache.nutch.parse.ParseText
 
getThrownError() - Method in class org.apache.nutch.util.CommandRunner
 
getTimeout() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getTimeout() - Method in class org.apache.nutch.util.CommandRunner
 
getTimestamp() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
getTitle() - Method in class org.apache.nutch.parse.ParseData
The title of the page.
getToUrl() - Method in class org.apache.nutch.parse.Outlink
 
getTraceStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getTstamp() - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
getType() - Method in class org.apache.nutch.util.domain.TopLevelDomain
 
getTypes() - Method in class org.apache.nutch.crawl.NutchWritable
 
getUniqueKey() - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
getUrl() - Method in interface org.apache.nutch.net.protocols.Response
Returns the URL used to retrieve this response.
getUrl() - Method in exception org.apache.nutch.parse.ParserNotFound
 
getUrl() - Method in class org.apache.nutch.protocol.Content
The url fetched.
getUrl() - Method in class org.apache.nutch.protocol.http.HttpResponse
 
getUrl() - Method in class org.apache.nutch.protocol.httpclient.HttpResponse
 
getUrl() - Method in exception org.apache.nutch.protocol.ProtocolNotFound
 
getUrl() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
getUrl() - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
getUseHttp11() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getUserAgent() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
getUUID(Configuration) - Static method in class org.apache.nutch.util.NutchConfiguration
Retrieve a Nutch UUID of this configuration object, or null if the configuration was created elsewhere.
getValues() - Method in class org.apache.nutch.indexer.NutchField
 
getValues(String) - Method in class org.apache.nutch.metadata.Metadata
Get the values associated to a metadata name.
getValues(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
 
getVersion() - Method in class org.apache.nutch.parse.ParseData
 
getVersion() - Method in class org.apache.nutch.parse.ParseStatus
 
getVersion() - Method in class org.apache.nutch.plugin.PluginDescriptor
 
getWaitForExit() - Method in class org.apache.nutch.util.CommandRunner
 
getWarnStream(Log) - Static method in class org.apache.nutch.util.LogUtil
 
getWeight() - Method in class org.apache.nutch.indexer.NutchDocument
 
getWeight() - Method in class org.apache.nutch.indexer.NutchField
 
GONE - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Resource is gone.
guessEncoding(Content, String) - Method in class org.apache.nutch.util.EncodingDetector
Guess the encoding with the previously specified list of clues.
GZIPUtils - Class in org.apache.nutch.util
A collection of utility methods for working on GZIPed data.
GZIPUtils() - Constructor for class org.apache.nutch.util.GZIPUtils
 

H

HadoopFSUtil - Class in org.apache.nutch.util
 
HadoopFSUtil() - Constructor for class org.apache.nutch.util.HadoopFSUtil
 
handle(String, HttpServletRequest, HttpServletResponse, int) - Method in class org.apache.nutch.tools.proxy.AbstractTestbedHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.AbstractTestbedHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.DelayHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.FakeHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.LogDebugHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.NotFoundHandler
 
handle(Request, HttpServletResponse, String, int) - Method in class org.apache.nutch.tools.proxy.SegmentHandler
 
hasCopy(String) - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
hasDbStatus(CrawlDatum) - Static method in class org.apache.nutch.crawl.CrawlDatum
 
hasFetchStatus(CrawlDatum) - Static method in class org.apache.nutch.crawl.CrawlDatum
 
hashCode() - Method in class org.apache.nutch.crawl.CrawlDatum
 
hashCode() - Method in class org.apache.nutch.crawl.Inlink
 
hashCode() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
hashCode() - Method in class org.apache.nutch.protocol.httpclient.DummySSLProtocolSocketFactory
 
hasNext() - Method in class org.apache.nutch.util.NodeWalker
Returns true if there are more nodes on the current stack.
HTMLLanguageParser - Class in org.apache.nutch.analysis.lang
Adds metadata identifying language of document if found We could also run statistical analysis here but we'd miss all other formats
HTMLLanguageParser() - Constructor for class org.apache.nutch.analysis.lang.HTMLLanguageParser
 
HTMLMetaTags - Class in org.apache.nutch.parse
This class holds the information about HTML "meta" tags extracted from a page.
HTMLMetaTags() - Constructor for class org.apache.nutch.parse.HTMLMetaTags
 
HtmlParseFilter - Interface in org.apache.nutch.parse
Extension point for DOM-based HTML parsers.
HTMLPARSEFILTER_ORDER - Static variable in class org.apache.nutch.parse.HtmlParseFilters
 
HtmlParseFilters - Class in org.apache.nutch.parse
Creates and caches HtmlParseFilter implementing plugins.
HtmlParseFilters(Configuration) - Constructor for class org.apache.nutch.parse.HtmlParseFilters
 
Http - Class in org.apache.nutch.protocol.http
 
Http() - Constructor for class org.apache.nutch.protocol.http.Http
 
Http - Class in org.apache.nutch.protocol.httpclient
This class is a protocol plugin that configures an HTTP client for Basic, Digest and NTLM authentication schemes for web server as well as proxy server.
Http() - Constructor for class org.apache.nutch.protocol.httpclient.Http
Constructs this plugin.
HttpAuthentication - Interface in org.apache.nutch.protocol.httpclient
The base level of services required for Http Authentication
HttpAuthenticationException - Exception in org.apache.nutch.protocol.httpclient
Can be used to identify problems during creation of Authentication objects.
HttpAuthenticationException() - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
Constructs a new exception with null as its detail message.
HttpAuthenticationException(String) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
Constructs a new exception with the specified detail message.
HttpAuthenticationException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
Constructs a new exception with the specified message and cause.
HttpAuthenticationException(Throwable) - Constructor for exception org.apache.nutch.protocol.httpclient.HttpAuthenticationException
Constructs a new exception with the specified cause and detail message from given clause if it is not null.
HttpAuthenticationFactory - Class in org.apache.nutch.protocol.httpclient
Provides the Http protocol implementation with the ability to authenticate when prompted.
HttpAuthenticationFactory(Configuration) - Constructor for class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
 
HttpBase - Class in org.apache.nutch.protocol.http.api
 
HttpBase() - Constructor for class org.apache.nutch.protocol.http.api.HttpBase
Creates a new instance of HttpBase
HttpBase(Log) - Constructor for class org.apache.nutch.protocol.http.api.HttpBase
Creates a new instance of HttpBase
HttpBasicAuthentication - Class in org.apache.nutch.protocol.httpclient
Implementation of RFC 2617 Basic Authentication.
HttpBasicAuthentication(String, Configuration) - Constructor for class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
Construct an HttpBasicAuthentication for the given challenge parameters.
HttpDateFormat - Class in org.apache.nutch.net.protocols
class to handle HTTP dates.
HttpDateFormat() - Constructor for class org.apache.nutch.net.protocols.HttpDateFormat
 
HttpException - Exception in org.apache.nutch.protocol.http.api
 
HttpException() - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
 
HttpException(String) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
 
HttpException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
 
HttpException(Throwable) - Constructor for exception org.apache.nutch.protocol.http.api.HttpException
 
HttpHeaders - Interface in org.apache.nutch.metadata
A collection of HTTP header names.
HttpResponse - Class in org.apache.nutch.protocol.http
An HTTP response.
HttpResponse(HttpBase, URL, CrawlDatum) - Constructor for class org.apache.nutch.protocol.http.HttpResponse
 
HttpResponse - Class in org.apache.nutch.protocol.httpclient
An HTTP response.

I

ID_FIELD - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
IDENTIFIER - Static variable in interface org.apache.nutch.metadata.DublinCore
Recommended best practice is to identify the resource by means of a string or number conforming to a formal identification system.
identify(String) - Method in class org.apache.nutch.analysis.lang.LanguageIdentifier
Identify language of a content.
identify(StringBuilder) - Method in class org.apache.nutch.analysis.lang.LanguageIdentifier
Identify language of a content.
identify(InputStream) - Method in class org.apache.nutch.analysis.lang.LanguageIdentifier
Identify language from input stream.
identify(InputStream, String) - Method in class org.apache.nutch.analysis.lang.LanguageIdentifier
Identify language from input stream.
in - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
IndexerMapReduce - Class in org.apache.nutch.indexer
 
IndexerMapReduce() - Constructor for class org.apache.nutch.indexer.IndexerMapReduce
 
IndexerOutputFormat - Class in org.apache.nutch.indexer
 
IndexerOutputFormat() - Constructor for class org.apache.nutch.indexer.IndexerOutputFormat
 
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Dampen the boost value by scorePower.
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in interface org.apache.nutch.scoring.ScoringFilter
This method calculates a Lucene document boost.
indexerScore(Text, NutchDocument, CrawlDatum, CrawlDatum, Parse, Inlinks, float) - Method in class org.apache.nutch.scoring.ScoringFilters
 
IndexingException - Exception in org.apache.nutch.indexer
 
IndexingException() - Constructor for exception org.apache.nutch.indexer.IndexingException
 
IndexingException(String) - Constructor for exception org.apache.nutch.indexer.IndexingException
 
IndexingException(String, Throwable) - Constructor for exception org.apache.nutch.indexer.IndexingException
 
IndexingException(Throwable) - Constructor for exception org.apache.nutch.indexer.IndexingException
 
IndexingFilter - Interface in org.apache.nutch.indexer
Extension point for indexing.
INDEXINGFILTER_ORDER - Static variable in class org.apache.nutch.indexer.IndexingFilters
 
IndexingFilters - Class in org.apache.nutch.indexer
Creates and caches IndexingFilter implementing plugins.
IndexingFilters(Configuration) - Constructor for class org.apache.nutch.indexer.IndexingFilters
 
indexSolr(String, Path, Path, List<Path>) - Method in class org.apache.nutch.indexer.solr.SolrIndexer
 
inflate(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
Returns an inflated copy of the input array.
inflateBestEffort(byte[]) - Static method in class org.apache.nutch.util.DeflateUtils
Returns an inflated copy of the input array.
inflateBestEffort(byte[], int) - Static method in class org.apache.nutch.util.DeflateUtils
Returns an inflated copy of the input array, truncated to sizeLimit bytes, if necessary.
init(Path) - Method in class org.apache.nutch.crawl.LinkDbReader
 
init(FilterConfig) - Method in class org.apache.nutch.tools.proxy.LogDebugHandler
 
initializeSchedule(Text, CrawlDatum) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
Initialize fetch schedule related data.
initializeSchedule(Text, CrawlDatum) - Method in interface org.apache.nutch.crawl.FetchSchedule
Initialize fetch schedule related data.
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Set to 0.0f (unknown value) - inlink contributions will bring it to a correct level.
initialScore(Text, CrawlDatum) - Method in interface org.apache.nutch.scoring.ScoringFilter
Set an initial score for newly discovered pages.
initialScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.ScoringFilters
Calculate a new initial score, used when adding newly discovered pages.
initMRJob(Path, Path, Collection<Path>, JobConf) - Static method in class org.apache.nutch.indexer.IndexerMapReduce
 
inject(Path, Path) - Method in class org.apache.nutch.crawl.Injector
 
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
 
injectedScore(Text, CrawlDatum) - Method in interface org.apache.nutch.scoring.ScoringFilter
Set an initial score for newly injected pages.
injectedScore(Text, CrawlDatum) - Method in class org.apache.nutch.scoring.ScoringFilters
Calculate a new initial score, used when injecting new pages.
Injector - Class in org.apache.nutch.crawl
This class takes a flat file of URLs and adds them to the of pages to be crawled.
Injector() - Constructor for class org.apache.nutch.crawl.Injector
 
Injector(Configuration) - Constructor for class org.apache.nutch.crawl.Injector
 
Injector.InjectMapper - Class in org.apache.nutch.crawl
Normalize and filter injected urls.
Injector.InjectMapper() - Constructor for class org.apache.nutch.crawl.Injector.InjectMapper
 
Injector.InjectReducer - Class in org.apache.nutch.crawl
Combine multiple new entries for a url.
Injector.InjectReducer() - Constructor for class org.apache.nutch.crawl.Injector.InjectReducer
 
Inlink - Class in org.apache.nutch.crawl
 
Inlink() - Constructor for class org.apache.nutch.crawl.Inlink
 
Inlink(String, String) - Constructor for class org.apache.nutch.crawl.Inlink
 
INLINK - Static variable in class org.apache.nutch.scoring.webgraph.LinkDatum
 
INLINK_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
 
Inlinks - Class in org.apache.nutch.crawl
A list of Inlinks.
Inlinks() - Constructor for class org.apache.nutch.crawl.Inlinks
 
install(JobConf, Path) - Static method in class org.apache.nutch.crawl.CrawlDb
 
install(JobConf, Path) - Static method in class org.apache.nutch.crawl.LinkDb
 
invert(Path, Path, boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDb
 
invert(Path, Path[], boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDb
 
isAllowed(URL) - Method in class org.apache.nutch.protocol.EmptyRobotRules
 
isAllowed(HttpBase, URL) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
isAllowed(URL) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Returns false if the robots.txt file prohibits us from accessing the given url, or true otherwise.
isAllowed(String) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Returns false if the robots.txt file prohibits us from accessing the given path, or true otherwise.
isAllowed(URL) - Method in interface org.apache.nutch.protocol.RobotRules
Returns false if the robots.txt file prohibits us from accessing the given url, or true otherwise.
isCanonical() - Method in interface org.apache.nutch.parse.Parse
Indicates if the parse is coming from a url or a sub-url
isCanonical() - Method in class org.apache.nutch.parse.ParseImpl
 
isClientTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
 
isDomainSuffix(String) - Method in class org.apache.nutch.util.domain.DomainSuffixes
return whether the extension is a registered domain entry
isEmpty() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
isEmpty() - Method in class org.apache.nutch.parse.ParseResult
Checks whether the result is empty.
isEmpty(String) - Static method in class org.apache.nutch.util.StringUtil
Checks if a string is empty (ie is null or empty).
isFound() - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
isMagic(byte[]) - Static method in class org.apache.nutch.tools.arc.ArcRecordReader
Returns true if the byte array passed matches the gzip header magic number.
isMultiValued(String) - Method in class org.apache.nutch.metadata.Metadata
Returns true if named value is multivalued.
isParsing(Configuration) - Static method in class org.apache.nutch.fetcher.Fetcher
 
isParsing(Configuration) - Static method in class org.apache.nutch.fetcher.OldFetcher
 
isPermanentFailure() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
isRemoteVerificationEnabled() - Method in class org.apache.nutch.protocol.ftp.Client
Return whether or not verification of the remote host participating in data connections is enabled.
isSameDomainName(URL, URL) - Static method in class org.apache.nutch.util.URLUtil
Returns whether the given urls have the same domain name.
isSameDomainName(String, String) - Static method in class org.apache.nutch.util.URLUtil
Returns whether the given urls have the same domain name.
isServerTrusted(X509Certificate[]) - Method in class org.apache.nutch.protocol.httpclient.DummyX509TrustManager
 
isStoringContent(Configuration) - Static method in class org.apache.nutch.fetcher.Fetcher
 
isStoringContent(Configuration) - Static method in class org.apache.nutch.fetcher.OldFetcher
 
isSuccess() - Method in class org.apache.nutch.parse.ParseResult
A convenience method which returns true only if all parses are successful.
isSuccess() - Method in class org.apache.nutch.parse.ParseStatus
A convenience method.
isSuccess() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
isTransientFailure() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
iterator() - Method in class org.apache.nutch.crawl.Inlinks
 
iterator() - Method in class org.apache.nutch.indexer.NutchDocument
Iterate over all fields.
iterator() - Method in class org.apache.nutch.parse.ParseResult
Iterate over all entries in the <url, Parse> map.

J

JSParseFilter - Class in org.apache.nutch.parse.js
This class is a heuristic link extractor for JavaScript files and code snippets.
JSParseFilter() - Constructor for class org.apache.nutch.parse.js.JSParseFilter
 

K

keySet() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
KEYWORDS - Static variable in interface org.apache.nutch.metadata.Office
 

L

LANGUAGE - Static variable in interface org.apache.nutch.metadata.DublinCore
A language of the intellectual content of the resource.
LanguageIdentifier - Class in org.apache.nutch.analysis.lang
Identify the language of a content, based on statistical analysis.
LanguageIdentifier(Configuration) - Constructor for class org.apache.nutch.analysis.lang.LanguageIdentifier
Constructs a new Language Identifier.
LanguageIndexingFilter - Class in org.apache.nutch.analysis.lang
An IndexingFilter that add a lang (language) field to the document.
LanguageIndexingFilter() - Constructor for class org.apache.nutch.analysis.lang.LanguageIndexingFilter
Constructs a new Language Indexing Filter.
LAST_AUTHOR - Static variable in interface org.apache.nutch.metadata.Office
 
LAST_MODIFIED - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
LAST_PRINTED - Static variable in interface org.apache.nutch.metadata.Office
 
LAST_SAVED - Static variable in interface org.apache.nutch.metadata.Office
 
leftPad(String, int) - Static method in class org.apache.nutch.util.StringUtil
Returns a copy of s padded with leading spaces so that it's length is length.
LICENSE_LOCATION - Static variable in interface org.apache.nutch.metadata.CreativeCommons
 
LICENSE_URL - Static variable in interface org.apache.nutch.metadata.CreativeCommons
 
LinkDatum - Class in org.apache.nutch.scoring.webgraph
A class for holding link information including the url, anchor text, a score, the timestamp of the link and a link type.
LinkDatum() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
Default constructor, no url, timestamp, score, or link type.
LinkDatum(String) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
Creates a LinkDatum with a given url.
LinkDatum(String, String) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
Creates a LinkDatum with a url and an anchor text.
LinkDatum(String, String, long) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDatum
 
LinkDb - Class in org.apache.nutch.crawl
Maintains an inverted link map, listing incoming links for each url.
LinkDb() - Constructor for class org.apache.nutch.crawl.LinkDb
 
LinkDb(Configuration) - Constructor for class org.apache.nutch.crawl.LinkDb
 
LinkDbFilter - Class in org.apache.nutch.crawl
This class provides a way to separate the URL normalization and filtering steps from the rest of LinkDb manipulation code.
LinkDbFilter() - Constructor for class org.apache.nutch.crawl.LinkDbFilter
 
LinkDbMerger - Class in org.apache.nutch.crawl
This tool merges several LinkDb-s into one, optionally filtering URLs through the current URLFilters, to skip prohibited URLs and links.
LinkDbMerger() - Constructor for class org.apache.nutch.crawl.LinkDbMerger
 
LinkDbMerger(Configuration) - Constructor for class org.apache.nutch.crawl.LinkDbMerger
 
LinkDbReader - Class in org.apache.nutch.crawl
.
LinkDbReader() - Constructor for class org.apache.nutch.crawl.LinkDbReader
 
LinkDbReader(Configuration, Path) - Constructor for class org.apache.nutch.crawl.LinkDbReader
 
LinkDumper - Class in org.apache.nutch.scoring.webgraph
The LinkDumper tool creates a database of node to inlink information that can be read using the nested Reader class.
LinkDumper() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper
 
LinkDumper.Inverter - Class in org.apache.nutch.scoring.webgraph
Inverts outlinks from the WebGraph to inlinks and attaches node information.
LinkDumper.Inverter() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
 
LinkDumper.LinkNode - Class in org.apache.nutch.scoring.webgraph
Bean class which holds url to node information.
LinkDumper.LinkNode() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
LinkDumper.LinkNode(String, Node) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
LinkDumper.LinkNodes - Class in org.apache.nutch.scoring.webgraph
Writable class which holds an array of LinkNode objects.
LinkDumper.LinkNodes() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
LinkDumper.LinkNodes(LinkDumper.LinkNode[]) - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
LinkDumper.Merger - Class in org.apache.nutch.scoring.webgraph
Merges LinkNode objects into a single array value per url.
LinkDumper.Merger() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
 
LinkDumper.Reader - Class in org.apache.nutch.scoring.webgraph
Reader class which will print out the url and all of its inlinks to system out.
LinkDumper.Reader() - Constructor for class org.apache.nutch.scoring.webgraph.LinkDumper.Reader
 
LinkRank - Class in org.apache.nutch.scoring.webgraph
 
LinkRank() - Constructor for class org.apache.nutch.scoring.webgraph.LinkRank
Default constructor.
LinkRank(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.LinkRank
Configurable constructor.
list(List<Path>, Writer) - Method in class org.apache.nutch.segment.SegmentReader
 
load(InputStream) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Loads a ngram profile from an InputStream (assumes UTF-8 encoded content)
LOCATION - Static variable in interface org.apache.nutch.metadata.HttpHeaders
 
LOCK_NAME - Static variable in class org.apache.nutch.crawl.CrawlDb
 
LOCK_NAME - Static variable in class org.apache.nutch.crawl.LinkDb
 
LOCK_NAME - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
 
LockUtil - Class in org.apache.nutch.util
Utility methods for handling application-level locking.
LockUtil() - Constructor for class org.apache.nutch.util.LockUtil
 
LOG - Static variable in class org.apache.nutch.analysis.lang.HTMLLanguageParser
 
LOG - Static variable in class org.apache.nutch.analysis.lang.NGramProfile
 
LOG - Static variable in class org.apache.nutch.crawl.Crawl
 
LOG - Static variable in class org.apache.nutch.crawl.CrawlDb
 
LOG - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
 
LOG - Static variable in class org.apache.nutch.crawl.CrawlDbReader
 
LOG - Static variable in class org.apache.nutch.crawl.CrawlDbReducer
 
LOG - Static variable in class org.apache.nutch.crawl.FetchScheduleFactory
 
LOG - Static variable in class org.apache.nutch.crawl.Generator
 
LOG - Static variable in class org.apache.nutch.crawl.Injector
 
LOG - Static variable in class org.apache.nutch.crawl.LinkDb
 
LOG - Static variable in class org.apache.nutch.crawl.LinkDbFilter
 
LOG - Static variable in class org.apache.nutch.crawl.LinkDbReader
 
LOG - Static variable in class org.apache.nutch.crawl.MapWritable
Deprecated.  
LOG - Static variable in class org.apache.nutch.fetcher.Fetcher
 
LOG - Static variable in class org.apache.nutch.fetcher.OldFetcher
 
LOG - Static variable in class org.apache.nutch.indexer.basic.BasicIndexingFilter
 
LOG - Static variable in class org.apache.nutch.indexer.IndexerMapReduce
 
LOG - Static variable in class org.apache.nutch.indexer.IndexingFilters
 
LOG - Static variable in class org.apache.nutch.indexer.more.MoreIndexingFilter
 
LOG - Static variable in class org.apache.nutch.indexer.solr.SolrClean
 
LOG - Static variable in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
LOG - Static variable in class org.apache.nutch.indexer.solr.SolrIndexer
 
LOG - Static variable in class org.apache.nutch.indexer.solr.SolrMappingReader
 
LOG - Static variable in class org.apache.nutch.microformats.reltag.RelTagParser
 
LOG - Static variable in class org.apache.nutch.net.URLNormalizers
 
LOG - Static variable in class org.apache.nutch.parse.ext.ExtParser
 
LOG - Static variable in class org.apache.nutch.parse.js.JSParseFilter
 
LOG - Static variable in class org.apache.nutch.parse.ParserChecker
 
LOG - Static variable in class org.apache.nutch.parse.ParseResult
 
LOG - Static variable in class org.apache.nutch.parse.ParserFactory
 
LOG - Static variable in class org.apache.nutch.parse.ParseSegment
 
LOG - Static variable in class org.apache.nutch.parse.ParseUtil
 
LOG - Static variable in class org.apache.nutch.parse.swf.SWFParser
 
LOG - Static variable in class org.apache.nutch.parse.tika.TikaParser
 
LOG - Static variable in class org.apache.nutch.parse.zip.ZipTextExtractor
 
LOG - Static variable in class org.apache.nutch.plugin.PluginDescriptor
 
LOG - Static variable in class org.apache.nutch.plugin.PluginManifestParser
 
LOG - Static variable in class org.apache.nutch.plugin.PluginRepository
 
LOG - Static variable in class org.apache.nutch.protocol.file.File
 
LOG - Static variable in class org.apache.nutch.protocol.ftp.Ftp
 
LOG - Static variable in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
LOG - Static variable in class org.apache.nutch.protocol.http.Http
 
LOG - Static variable in class org.apache.nutch.protocol.httpclient.Http
 
LOG - Static variable in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
 
LOG - Static variable in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
 
LOG - Static variable in class org.apache.nutch.protocol.ProtocolFactory
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.LinkDumper
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.LinkRank
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.Loops
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.NodeDumper
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.ScoreUpdater
 
LOG - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
 
LOG - Static variable in class org.apache.nutch.segment.SegmentReader
 
LOG - Static variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
LOG - Static variable in class org.apache.nutch.tools.arc.ArcSegmentCreator
 
LOG - Static variable in class org.apache.nutch.tools.CrawlDBScanner
 
LOG - Static variable in class org.apache.nutch.tools.DmozParser
 
LOG - Static variable in class org.apache.nutch.tools.ResolveUrls
 
LOG - Static variable in class org.apache.nutch.util.EncodingDetector
 
LOG - Static variable in class org.creativecommons.nutch.CCIndexingFilter
 
LOG - Static variable in class org.creativecommons.nutch.CCParseFilter
 
logConf() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
LogDebugHandler - Class in org.apache.nutch.tools.proxy
 
LogDebugHandler() - Constructor for class org.apache.nutch.tools.proxy.LogDebugHandler
 
login(String, String) - Method in class org.apache.nutch.protocol.ftp.Client
Login to the FTP server using the provided username and password.
logout() - Method in class org.apache.nutch.protocol.ftp.Client
Logout of the FTP server by sending the QUIT command.
LogUtil - Class in org.apache.nutch.util
Utility class for logging.
LogUtil() - Constructor for class org.apache.nutch.util.LogUtil
 
longestMatch(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
Returns the longest prefix of input that is matched, or null if no match exists.
longestMatch(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
Returns the longest suffix of input that is matched, or null if no match exists.
longestMatch(String) - Method in class org.apache.nutch.util.TrieStringMatcher
Returns the longest substring of input that is matched by a pattern in the trie, or null if no match exists.
LoopReader - Class in org.apache.nutch.scoring.webgraph
The LoopReader tool prints the loopset information for a single url.
LoopReader() - Constructor for class org.apache.nutch.scoring.webgraph.LoopReader
 
LoopReader(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.LoopReader
 
Loops - Class in org.apache.nutch.scoring.webgraph
The Loops job identifies cycles of loops inside of the web graph.
Loops() - Constructor for class org.apache.nutch.scoring.webgraph.Loops
 
Loops.Finalizer - Class in org.apache.nutch.scoring.webgraph
Finishes the Loops job by aggregating and collecting and found routes.
Loops.Finalizer() - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Finalizer
Default constructor.
Loops.Finalizer(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Finalizer
Configurable constructor.
Loops.Initializer - Class in org.apache.nutch.scoring.webgraph
Initializes the Loop routes.
Loops.Initializer() - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Initializer
Default constructor.
Loops.Initializer(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Initializer
Configurable constructor.
Loops.Looper - Class in org.apache.nutch.scoring.webgraph
Follows a route path looking for the start url of the route.
Loops.Looper() - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Looper
Default constructor.
Loops.Looper(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Looper
Configurable constructor.
Loops.LoopSet - Class in org.apache.nutch.scoring.webgraph
A set of loops.
Loops.LoopSet() - Constructor for class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
Loops.Route - Class in org.apache.nutch.scoring.webgraph
A link path or route looking to identify a link cycle.
Loops.Route() - Constructor for class org.apache.nutch.scoring.webgraph.Loops.Route
 
LOOPS_DIR - Static variable in class org.apache.nutch.scoring.webgraph.Loops
 

M

main(String[]) - Static method in class org.apache.nutch.analysis.lang.LanguageIdentifier
Main method used for command line process.
main(String[]) - Static method in class org.apache.nutch.analysis.lang.NGramProfile
main method used for testing only
main(String[]) - Static method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
 
main(String[]) - Static method in class org.apache.nutch.crawl.Crawl
 
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDb
 
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDbMerger
 
main(String[]) - Static method in class org.apache.nutch.crawl.CrawlDbReader
 
main(String[]) - Static method in class org.apache.nutch.crawl.Generator
Generate a fetchlist from the crawldb.
main(String[]) - Static method in class org.apache.nutch.crawl.Injector
 
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDb
 
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDbMerger
 
main(String[]) - Static method in class org.apache.nutch.crawl.LinkDbReader
 
main(String[]) - Static method in class org.apache.nutch.crawl.TextProfileSignature
 
main(String[]) - Static method in class org.apache.nutch.fetcher.Fetcher
Run the fetcher.
main(String[]) - Static method in class org.apache.nutch.fetcher.OldFetcher
Run the fetcher.
main(String[]) - Static method in class org.apache.nutch.indexer.solr.SolrClean
 
main(String[]) - Static method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
main(String[]) - Static method in class org.apache.nutch.indexer.solr.SolrIndexer
 
main(String[]) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
 
main(String[]) - Static method in class org.apache.nutch.net.URLFilterChecker
 
main(String[]) - Static method in class org.apache.nutch.net.URLNormalizerChecker
 
main(String[]) - Static method in class org.apache.nutch.parse.js.JSParseFilter
 
main(String[]) - Static method in class org.apache.nutch.parse.ParseData
 
main(String[]) - Static method in class org.apache.nutch.parse.ParserChecker
 
main(String[]) - Static method in class org.apache.nutch.parse.ParseSegment
 
main(String[]) - Static method in class org.apache.nutch.parse.ParseText
 
main(String[]) - Static method in class org.apache.nutch.parse.swf.SWFParser
Arguments are: 0.
main(String[]) - Static method in class org.apache.nutch.plugin.PluginRepository
Loads all necessary dependencies for a selected plugin, and then runs one of the classes' main() method.
main(String[]) - Static method in class org.apache.nutch.protocol.Content
 
main(String[]) - Static method in class org.apache.nutch.protocol.file.File
For debugging.
main(String[]) - Static method in class org.apache.nutch.protocol.ftp.Ftp
For debugging.
main(HttpBase, String[]) - Static method in class org.apache.nutch.protocol.http.api.HttpBase
 
main(String[]) - Static method in class org.apache.nutch.protocol.http.api.RobotRulesParser
command-line main for testing
main(String[]) - Static method in class org.apache.nutch.protocol.http.Http
 
main(String[]) - Static method in class org.apache.nutch.protocol.httpclient.Http
Main method.
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkDumper
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkDumper.Reader
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LinkRank
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.LoopReader
Runs the LoopReader tool.
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.Loops
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.NodeDumper
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.NodeReader
Runs the NodeReader tool.
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
 
main(String[]) - Static method in class org.apache.nutch.scoring.webgraph.WebGraph
 
main(String[]) - Static method in class org.apache.nutch.segment.SegmentMerger
 
main(String[]) - Static method in class org.apache.nutch.segment.SegmentReader
 
main(String[]) - Static method in class org.apache.nutch.tools.arc.ArcSegmentCreator
 
main(String[]) - Static method in class org.apache.nutch.tools.Benchmark
 
main(String[]) - Static method in class org.apache.nutch.tools.CrawlDBScanner
 
main(String[]) - Static method in class org.apache.nutch.tools.DmozParser
Command-line access.
main(String[]) - Static method in class org.apache.nutch.tools.FreeGenerator
 
main(String[]) - Static method in class org.apache.nutch.tools.proxy.TestbedProxy
 
main(String[]) - Static method in class org.apache.nutch.tools.ResolveUrls
Runs the resolve urls tool.
main(RegexURLFilterBase, String[]) - Static method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Filter the standard input using a RegexURLFilterBase.
main(String[]) - Static method in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
main(String[]) - Static method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
main(String[]) - Static method in class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
main(String[]) - Static method in class org.apache.nutch.util.CommandRunner
 
main(String[]) - Static method in class org.apache.nutch.util.domain.DomainStatistics
 
main(String[]) - Static method in class org.apache.nutch.util.EncodingDetector
 
main(String[]) - Static method in class org.apache.nutch.util.PrefixStringMatcher
 
main(String[]) - Static method in class org.apache.nutch.util.StringUtil
 
main(String[]) - Static method in class org.apache.nutch.util.SuffixStringMatcher
 
main(String[]) - Static method in class org.apache.nutch.util.URLUtil
For testing
majorCodes - Static variable in class org.apache.nutch.parse.ParseStatus
 
makeIOException(SolrServerException) - Static method in class org.apache.nutch.indexer.solr.SolrWriter
 
map(Text, CrawlDatum, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbFilter
 
map(Text, CrawlDatum, OutputCollector<Text, LongWritable>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatMapper
 
map(Text, CrawlDatum, OutputCollector<FloatWritable, Text>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNMapper
 
map(Text, CrawlDatum, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater
 
map(Text, CrawlDatum, OutputCollector<FloatWritable, Generator.SelectorEntry>, Reporter) - Method in class org.apache.nutch.crawl.Generator.Selector
Select & invert subset due for fetch.
map(FloatWritable, Generator.SelectorEntry, OutputCollector<Text, Generator.SelectorEntry>, Reporter) - Method in class org.apache.nutch.crawl.Generator.SelectorInverseMapper
 
map(WritableComparable, Text, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.Injector.InjectMapper
 
map(Text, ParseData, OutputCollector<Text, Inlinks>, Reporter) - Method in class org.apache.nutch.crawl.LinkDb
 
map(Text, Inlinks, OutputCollector<Text, Inlinks>, Reporter) - Method in class org.apache.nutch.crawl.LinkDbFilter
 
map(Text, Writable, OutputCollector<Text, NutchWritable>, Reporter) - Method in class org.apache.nutch.indexer.IndexerMapReduce
 
map(Text, CrawlDatum, OutputCollector<ByteWritable, Text>, Reporter) - Method in class org.apache.nutch.indexer.solr.SolrClean.DBFilter
 
map(WritableComparable, Content, OutputCollector<Text, ParseImpl>, Reporter) - Method in class org.apache.nutch.parse.ParseSegment
 
map(Text, Writable, OutputCollector<Text, ObjectWritable>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
Wraps all values in ObjectWritables.
map(Text, Loops.Route, OutputCollector<Text, Loops.Route>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Finalizer
Maps out and found routes, those will be the link cycles.
map(Text, Writable, OutputCollector<Text, ObjectWritable>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Initializer
Wraps values in ObjectWritable.
map(Text, Writable, OutputCollector<Text, ObjectWritable>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Looper
Wrap values in ObjectWritable.
map(Text, Node, OutputCollector<FloatWritable, Text>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
Outputs the url with the appropriate number of inlinks, outlinks, or for score.
map(Text, Writable, OutputCollector<Text, ObjectWritable>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
Changes input into ObjectWritables.
map(Text, Writable, OutputCollector<Text, LinkDatum>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
Passes through existing LinkDatum objects from an existing OutlinkDb and maps out new LinkDatum objects from new crawls ParseData.
map(Text, MetaWrapper, OutputCollector<Text, MetaWrapper>, Reporter) - Method in class org.apache.nutch.segment.SegmentMerger
 
map(WritableComparable, Writable, OutputCollector<Text, NutchWritable>, Reporter) - Method in class org.apache.nutch.segment.SegmentReader.InputCompatMapper
 
map(Text, BytesWritable, OutputCollector<Text, NutchWritable>, Reporter) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
Runs the Map job to translate an arc record into output for Nutch segments.
map(Text, CrawlDatum, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.tools.CrawlDBScanner
 
map(WritableComparable, Text, OutputCollector<Text, Generator.SelectorEntry>, Reporter) - Method in class org.apache.nutch.tools.FreeGenerator.FG
 
map(Text, CrawlDatum, OutputCollector<Text, LongWritable>, Reporter) - Method in class org.apache.nutch.util.domain.DomainStatistics
 
mapCopyKey(String) - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
mapKey(String) - Method in class org.apache.nutch.indexer.solr.SolrMappingReader
 
MAPPING_FILE - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
MapWritable - Class in org.apache.nutch.crawl
Deprecated. Use org.apache.hadoop.io.MapWritable instead.
MapWritable() - Constructor for class org.apache.nutch.crawl.MapWritable
Deprecated.  
MapWritable(MapWritable) - Constructor for class org.apache.nutch.crawl.MapWritable
Deprecated. Copy constructor.
match(String) - Method in class org.apache.nutch.urlfilter.api.RegexRule
Checks if a url matches this rule.
matchChar(TrieStringMatcher.TrieNode, String, int) - Method in class org.apache.nutch.util.TrieStringMatcher
Returns the next TrieStringMatcher.TrieNode visited, given that you are at node, and the the next character in the input is the idx'th character of s.
matches(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
Returns true if the given String is matched by a prefix in the trie
matches(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
Returns true if the given String is matched by a suffix in the trie
matches(String) - Method in class org.apache.nutch.util.TrieStringMatcher
Returns true if the given String is matched by a pattern in the trie
maxContent - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The length limit for downloaded content, in bytes.
maxCrawlDelay - Variable in class org.apache.nutch.protocol.http.api.HttpBase
Skip page if Crawl-Delay longer than this value.
maxInterval - Variable in class org.apache.nutch.crawl.AbstractFetchSchedule
 
MD5Signature - Class in org.apache.nutch.crawl
Default implementation of a page signature.
MD5Signature() - Constructor for class org.apache.nutch.crawl.MD5Signature
 
merge(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDbMerger
 
merge(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.LinkDbMerger
 
merge(Path, Path[], boolean, boolean, long) - Method in class org.apache.nutch.segment.SegmentMerger
 
Metadata - Class in org.apache.nutch.metadata
A multi-valued metadata container.
Metadata() - Constructor for class org.apache.nutch.metadata.Metadata
Constructs a new, empty metadata.
MetaWrapper - Class in org.apache.nutch.metadata
This is a simple decorator that adds metadata to any Writable-s that can be serialized by NutchWritable.
MetaWrapper() - Constructor for class org.apache.nutch.metadata.MetaWrapper
 
MetaWrapper(Writable, Configuration) - Constructor for class org.apache.nutch.metadata.MetaWrapper
 
MetaWrapper(Metadata, Writable, Configuration) - Constructor for class org.apache.nutch.metadata.MetaWrapper
 
MimeUtil - Class in org.apache.nutch.util
 
MimeUtil(Configuration) - Constructor for class org.apache.nutch.util.MimeUtil
 
MIN_CONFIDENCE_KEY - Static variable in class org.apache.nutch.util.EncodingDetector
 
MissingDependencyException - Exception in org.apache.nutch.plugin
MissingDependencyException will be thrown if a plugin dependency cannot be found.
MissingDependencyException(Throwable) - Constructor for exception org.apache.nutch.plugin.MissingDependencyException
 
MissingDependencyException(String) - Constructor for exception org.apache.nutch.plugin.MissingDependencyException
 
MODIFIED - Static variable in interface org.apache.nutch.metadata.DublinCore
Date on which the resource was changed.
MoreIndexingFilter - Class in org.apache.nutch.indexer.more
Add (or reset) a few metaData properties as respective fields (if they are available), so that they can be displayed by more.jsp (called by search.jsp).
MoreIndexingFilter() - Constructor for class org.apache.nutch.indexer.more.MoreIndexingFilter
 
MOVED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Resource has moved permanently.

N

names() - Method in class org.apache.nutch.metadata.Metadata
Returns an array of the names contained in the metadata.
next(Text, BytesWritable) - Method in class org.apache.nutch.tools.arc.ArcRecordReader
Returns true if the next record in the split is read into the key and value pair.
nextNode() - Method in class org.apache.nutch.util.NodeWalker
Returns the next Node on the stack and pushes all of its children onto the stack, allowing us to walk the node tree without the use of recursion.
NGramProfile - Class in org.apache.nutch.analysis.lang
This class runs a ngram analysis over submitted text, results might be used for automatic language identifiaction.
NGramProfile(String, int, int) - Constructor for class org.apache.nutch.analysis.lang.NGramProfile
Construct a new ngram profile
NO_THRESHOLD - Static variable in class org.apache.nutch.util.EncodingDetector
 
Node - Class in org.apache.nutch.scoring.webgraph
A class which holds the number of inlinks and outlinks for a given url along with an inlink score from a link analysis program and any metadata.
Node() - Constructor for class org.apache.nutch.scoring.webgraph.Node
 
NODE_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
 
nodeChar - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
 
NodeDumper - Class in org.apache.nutch.scoring.webgraph
A tools that dumps out the top urls by number of inlinks, number of outlinks, or by score, to a text file.
NodeDumper() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper
 
NodeDumper.Sorter - Class in org.apache.nutch.scoring.webgraph
Outputs the top urls sorted in descending order.
NodeDumper.Sorter() - Constructor for class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
 
NodeReader - Class in org.apache.nutch.scoring.webgraph
Reads and prints to system out information for a single node from the NodeDb in the WebGraph.
NodeReader() - Constructor for class org.apache.nutch.scoring.webgraph.NodeReader
 
NodeReader(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.NodeReader
 
NodeWalker - Class in org.apache.nutch.util
A utility class that allows the walking of any DOM tree using a stack instead of recursion.
NodeWalker(Node) - Constructor for class org.apache.nutch.util.NodeWalker
Starts the Node tree from the root node.
normalize() - Method in class org.apache.nutch.analysis.lang.NGramProfile
Normalize the profile (calculates the ngrams frequencies)
normalize(String, String) - Method in interface org.apache.nutch.net.URLNormalizer
 
normalize(String, String) - Method in class org.apache.nutch.net.URLNormalizers
Normalize
NOTFETCHING - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Not fetching.
NOTFOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Resource was not found.
NotFoundHandler - Class in org.apache.nutch.tools.proxy
 
NotFoundHandler() - Constructor for class org.apache.nutch.tools.proxy.NotFoundHandler
 
NOTMODIFIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Unchanged since the last fetch.
NOTPARSED - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing was not performed.
Nutch - Interface in org.apache.nutch.metadata
A collection of Nutch internal metadata constants.
NutchConfiguration - Class in org.apache.nutch.util
Utility to create Hadoop Configurations that include Nutch-specific resources.
NutchDocument - Class in org.apache.nutch.indexer
A NutchDocument is the unit of indexing.
NutchDocument() - Constructor for class org.apache.nutch.indexer.NutchDocument
 
nutchFetchIntervalMDName - Static variable in class org.apache.nutch.crawl.Injector
metadata key reserved for setting a custom fetchInterval for a specific URL
NutchField - Class in org.apache.nutch.indexer
This class represents a multi-valued field with a weight.
NutchField() - Constructor for class org.apache.nutch.indexer.NutchField
 
NutchField(Object) - Constructor for class org.apache.nutch.indexer.NutchField
 
NutchField(Object, float) - Constructor for class org.apache.nutch.indexer.NutchField
 
NutchIndexWriter - Interface in org.apache.nutch.indexer
 
NutchIndexWriterFactory - Class in org.apache.nutch.indexer
 
NutchIndexWriterFactory() - Constructor for class org.apache.nutch.indexer.NutchIndexWriterFactory
 
NutchJob - Class in org.apache.nutch.util
A JobConf for Nutch jobs.
NutchJob(Configuration) - Constructor for class org.apache.nutch.util.NutchJob
 
nutchScoreMDName - Static variable in class org.apache.nutch.crawl.Injector
metadata key reserved for setting a custom score for a specific URL
NutchWritable - Class in org.apache.nutch.crawl
 
NutchWritable() - Constructor for class org.apache.nutch.crawl.NutchWritable
 
NutchWritable(Writable) - Constructor for class org.apache.nutch.crawl.NutchWritable
 

O

ObjectCache - Class in org.apache.nutch.util
 
Office - Interface in org.apache.nutch.metadata
A collection of "Office" documents properties names.
OldFetcher - Class in org.apache.nutch.fetcher
The fetcher.
OldFetcher() - Constructor for class org.apache.nutch.fetcher.OldFetcher
 
OldFetcher(Configuration) - Constructor for class org.apache.nutch.fetcher.OldFetcher
 
OldFetcher.InputFormat - Class in org.apache.nutch.fetcher
 
OldFetcher.InputFormat() - Constructor for class org.apache.nutch.fetcher.OldFetcher.InputFormat
 
open(JobConf, String) - Method in interface org.apache.nutch.indexer.NutchIndexWriter
 
open(JobConf, String) - Method in class org.apache.nutch.indexer.solr.SolrWriter
 
OPICScoringFilter - Class in org.apache.nutch.scoring.opic
This plugin implements a variant of an Online Page Importance Computation (OPIC) score, described in this paper: Abiteboul, Serge and Preda, Mihai and Cobena, Gregory (2003), Adaptive On-Line Page Importance Computation .
OPICScoringFilter() - Constructor for class org.apache.nutch.scoring.opic.OPICScoringFilter
 
org.apache.nutch.analysis.lang - package org.apache.nutch.analysis.lang
Text document language identifier.
org.apache.nutch.crawl - package org.apache.nutch.crawl
Crawl control code.
org.apache.nutch.fetcher - package org.apache.nutch.fetcher
The Nutch robot.
org.apache.nutch.indexer - package org.apache.nutch.indexer
Maintain Lucene full-text indexes.
org.apache.nutch.indexer.basic - package org.apache.nutch.indexer.basic
A basic indexing plugin.
org.apache.nutch.indexer.more - package org.apache.nutch.indexer.more
A more indexing plugin.
org.apache.nutch.indexer.solr - package org.apache.nutch.indexer.solr
 
org.apache.nutch.metadata - package org.apache.nutch.metadata
A Multi-valued Metadata container, and set of constant fields for Nutch Metadata.
org.apache.nutch.microformats.reltag - package org.apache.nutch.microformats.reltag
A microformats Rel-Tag Parser/Indexer/Querier plugin.
org.apache.nutch.net - package org.apache.nutch.net
 
org.apache.nutch.net.protocols - package org.apache.nutch.net.protocols
 
org.apache.nutch.parse - package org.apache.nutch.parse
 
org.apache.nutch.parse.ext - package org.apache.nutch.parse.ext
 
org.apache.nutch.parse.js - package org.apache.nutch.parse.js
 
org.apache.nutch.parse.swf - package org.apache.nutch.parse.swf
 
org.apache.nutch.parse.tika - package org.apache.nutch.parse.tika
 
org.apache.nutch.parse.zip - package org.apache.nutch.parse.zip
 
org.apache.nutch.plugin - package org.apache.nutch.plugin
The Nutch Plugin System.
org.apache.nutch.protocol - package org.apache.nutch.protocol
 
org.apache.nutch.protocol.file - package org.apache.nutch.protocol.file
Protocol plugin which supports retrieving local file resources.
org.apache.nutch.protocol.ftp - package org.apache.nutch.protocol.ftp
Protocol plugin which supports retrieving documents via the ftp protocol.
org.apache.nutch.protocol.http - package org.apache.nutch.protocol.http
Protocol plugin which supports retrieving documents via the http protocol.
org.apache.nutch.protocol.http.api - package org.apache.nutch.protocol.http.api
Common API used by HTTP plugins (http, httpclient)
org.apache.nutch.protocol.httpclient - package org.apache.nutch.protocol.httpclient
Protocol plugin which supports retrieving documents via the HTTP and HTTPS protocols, optionally with Basic, Digest and NTLM authentication schemes for web server as well as proxy server.
org.apache.nutch.scoring - package org.apache.nutch.scoring
 
org.apache.nutch.scoring.opic - package org.apache.nutch.scoring.opic
 
org.apache.nutch.scoring.webgraph - package org.apache.nutch.scoring.webgraph
 
org.apache.nutch.segment - package org.apache.nutch.segment
 
org.apache.nutch.tools - package org.apache.nutch.tools
 
org.apache.nutch.tools.arc - package org.apache.nutch.tools.arc
 
org.apache.nutch.tools.proxy - package org.apache.nutch.tools.proxy
 
org.apache.nutch.urlfilter.api - package org.apache.nutch.urlfilter.api
 
org.apache.nutch.urlfilter.automaton - package org.apache.nutch.urlfilter.automaton
A url filter plugin based on dk.brics.automaton Finite-State Automata for JavaTM.
org.apache.nutch.urlfilter.prefix - package org.apache.nutch.urlfilter.prefix
A url filter plugin.
org.apache.nutch.urlfilter.regex - package org.apache.nutch.urlfilter.regex
A url filter plugin.
org.apache.nutch.util - package org.apache.nutch.util
 
org.apache.nutch.util.domain - package org.apache.nutch.util.domain
org.apache.nutch.util.domain
org.creativecommons.nutch - package org.creativecommons.nutch
Sample plugins that parse and index Creative Commons medadata.
ORIGINAL_CHAR_ENCODING - Static variable in interface org.apache.nutch.metadata.Nutch
 
Outlink - Class in org.apache.nutch.parse
 
Outlink() - Constructor for class org.apache.nutch.parse.Outlink
 
Outlink(String, String) - Constructor for class org.apache.nutch.parse.Outlink
 
OUTLINK - Static variable in class org.apache.nutch.scoring.webgraph.LinkDatum
 
OUTLINK_DIR - Static variable in class org.apache.nutch.scoring.webgraph.WebGraph
 
OutlinkExtractor - Class in org.apache.nutch.parse
Extractor to extract Outlinks / URLs from plain text using Regular Expressions.
OutlinkExtractor() - Constructor for class org.apache.nutch.parse.OutlinkExtractor
 

P

PAGE_COUNT - Static variable in interface org.apache.nutch.metadata.Office
 
Parse - Interface in org.apache.nutch.parse
The result of parsing a page's raw content.
parse(Path) - Method in class org.apache.nutch.parse.ParseSegment
 
parse(Content) - Method in class org.apache.nutch.parse.ParseUtil
Performs a parse by iterating through a List of preferred Parsers until a successful parse is performed and a Parse object is returned.
parse(String) - Static method in class org.apache.nutch.segment.SegmentPart
Create SegmentPart from a String in format "segmentName/partName".
PARSE_DIR_NAME - Static variable in class org.apache.nutch.crawl.CrawlDatum
 
parseByExtensionId(String, Content) - Method in class org.apache.nutch.parse.ParseUtil
Method parses a Content object using the Parser specified by the parameter extId, i.e., the Parser's extension ID.
parseCharacterEncoding(String) - Static method in class org.apache.nutch.util.EncodingDetector
Parse the character encoding from the specified content type header.
parsed - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
ParseData - Class in org.apache.nutch.parse
Data extracted from a page's content.
ParseData() - Constructor for class org.apache.nutch.parse.ParseData
 
ParseData(ParseStatus, String, Outlink[], Metadata) - Constructor for class org.apache.nutch.parse.ParseData
 
ParseData(ParseStatus, String, Outlink[], Metadata, Metadata) - Constructor for class org.apache.nutch.parse.ParseData
 
parseDmozFile(File, int, boolean, int, Pattern) - Method in class org.apache.nutch.tools.DmozParser
Iterate through all the items in this structured DMOZ file.
parseErrors - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
ParseException - Exception in org.apache.nutch.parse
 
ParseException() - Constructor for exception org.apache.nutch.parse.ParseException
 
ParseException(String) - Constructor for exception org.apache.nutch.parse.ParseException
 
ParseException(String, Throwable) - Constructor for exception org.apache.nutch.parse.ParseException
 
ParseException(Throwable) - Constructor for exception org.apache.nutch.parse.ParseException
 
ParseImpl - Class in org.apache.nutch.parse
The result of parsing a page's raw content.
ParseImpl() - Constructor for class org.apache.nutch.parse.ParseImpl
 
ParseImpl(Parse) - Constructor for class org.apache.nutch.parse.ParseImpl
 
ParseImpl(String, ParseData) - Constructor for class org.apache.nutch.parse.ParseImpl
 
ParseImpl(ParseText, ParseData) - Constructor for class org.apache.nutch.parse.ParseImpl
 
ParseImpl(ParseText, ParseData, boolean) - Constructor for class org.apache.nutch.parse.ParseImpl
 
ParseOutputFormat - Class in org.apache.nutch.parse
 
ParseOutputFormat() - Constructor for class org.apache.nutch.parse.ParseOutputFormat
 
parsePluginFolder(String[]) - Method in class org.apache.nutch.plugin.PluginManifestParser
Returns a list of all found plugin descriptors.
Parser - Interface in org.apache.nutch.parse
A parser for content generated by a Protocol implementation.
ParserChecker - Class in org.apache.nutch.parse
Parser checker, useful for testing parser.
ParserChecker() - Constructor for class org.apache.nutch.parse.ParserChecker
 
ParseResult - Class in org.apache.nutch.parse
A utility class that stores result of a parse.
ParseResult(String) - Constructor for class org.apache.nutch.parse.ParseResult
Create a container for parse results.
ParserFactory - Class in org.apache.nutch.parse
Creates and caches Parser plugins.
ParserFactory(Configuration) - Constructor for class org.apache.nutch.parse.ParserFactory
 
ParserNotFound - Exception in org.apache.nutch.parse
 
ParserNotFound(String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
 
ParserNotFound(String, String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
 
ParserNotFound(String, String, String) - Constructor for exception org.apache.nutch.parse.ParserNotFound
 
ParseSegment - Class in org.apache.nutch.parse
 
ParseSegment() - Constructor for class org.apache.nutch.parse.ParseSegment
 
ParseSegment(Configuration) - Constructor for class org.apache.nutch.parse.ParseSegment
 
ParseStatus - Class in org.apache.nutch.parse
 
ParseStatus() - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseStatus(int, int, String[]) - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseStatus(int) - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseStatus(int, String[]) - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseStatus(int, int) - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseStatus(int, int, String) - Constructor for class org.apache.nutch.parse.ParseStatus
Simplified constructor for passing just a text message.
ParseStatus(int, String) - Constructor for class org.apache.nutch.parse.ParseStatus
Simplified constructor for passing just a text message.
ParseStatus(Throwable) - Constructor for class org.apache.nutch.parse.ParseStatus
 
ParseText - Class in org.apache.nutch.parse
 
ParseText() - Constructor for class org.apache.nutch.parse.ParseText
 
ParseText(String) - Constructor for class org.apache.nutch.parse.ParseText
 
ParseUtil - Class in org.apache.nutch.parse
A Utility class containing methods to simply perform parsing utilities such as iterating through a preferred list of Parsers to obtain Parse objects.
ParseUtil(Configuration) - Constructor for class org.apache.nutch.parse.ParseUtil
 
PARTITION_MODE_DOMAIN - Static variable in class org.apache.nutch.crawl.URLPartitioner
 
PARTITION_MODE_HOST - Static variable in class org.apache.nutch.crawl.URLPartitioner
 
PARTITION_MODE_IP - Static variable in class org.apache.nutch.crawl.URLPartitioner
 
PARTITION_MODE_KEY - Static variable in class org.apache.nutch.crawl.URLPartitioner
 
partName - Variable in class org.apache.nutch.segment.SegmentPart
Name of the segment part (ie.
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Copy the value from Content metadata under Fetcher.SCORE_KEY to parseData.
passScoreAfterParsing(Text, Content, Parse) - Method in interface org.apache.nutch.scoring.ScoringFilter
Currently a part of score distribution is performed using only data coming from the parsing process.
passScoreAfterParsing(Text, Content, Parse) - Method in class org.apache.nutch.scoring.ScoringFilters
 
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Store a float value of CrawlDatum.getScore() under Fetcher.SCORE_KEY.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in interface org.apache.nutch.scoring.ScoringFilter
This method takes all relevant score information from the current datum (coming from a generated fetchlist) and stores it into Content metadata.
passScoreBeforeParsing(Text, CrawlDatum, Content) - Method in class org.apache.nutch.scoring.ScoringFilters
 
PERM_REFRESH_TIME - Static variable in class org.apache.nutch.fetcher.Fetcher
 
PERM_REFRESH_TIME - Static variable in class org.apache.nutch.fetcher.OldFetcher
 
Pluggable - Interface in org.apache.nutch.plugin
Defines the capability of a class to be plugged into Nutch.
Plugin - Class in org.apache.nutch.plugin
A nutch-plugin is an container for a set of custom logic that provide extensions to the nutch core functionality or another plugin that provides an API for extending.
Plugin(PluginDescriptor, Configuration) - Constructor for class org.apache.nutch.plugin.Plugin
Constructor
PluginClassLoader - Class in org.apache.nutch.plugin
The PluginClassLoader contains only classes of the runtime libraries setuped in the plugin manifest file and exported libraries of plugins that are required pluguin.
PluginClassLoader(URL[], ClassLoader) - Constructor for class org.apache.nutch.plugin.PluginClassLoader
Construtor
PluginDescriptor - Class in org.apache.nutch.plugin
The PluginDescriptor provide access to all meta information of a nutch-plugin, as well to the internationalizable resources and the plugin own classloader.
PluginDescriptor(String, String, String, String, String, String, Configuration) - Constructor for class org.apache.nutch.plugin.PluginDescriptor
Constructor
PluginManifestParser - Class in org.apache.nutch.plugin
The PluginManifestParser parser just parse the manifest file in all plugin directories.
PluginManifestParser(Configuration, PluginRepository) - Constructor for class org.apache.nutch.plugin.PluginManifestParser
 
PluginRepository - Class in org.apache.nutch.plugin
The plugin repositority is a registry of all plugins.
PluginRepository(Configuration) - Constructor for class org.apache.nutch.plugin.PluginRepository
 
PluginRuntimeException - Exception in org.apache.nutch.plugin
PluginRuntimeException will be thrown until a exception in the plugin managemnt occurs.
PluginRuntimeException(Throwable) - Constructor for exception org.apache.nutch.plugin.PluginRuntimeException
 
PluginRuntimeException(String) - Constructor for exception org.apache.nutch.plugin.PluginRuntimeException
 
pos - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
PrefixStringMatcher - Class in org.apache.nutch.util
A class for efficiently matching Strings against a set of prefixes.
PrefixStringMatcher(String[]) - Constructor for class org.apache.nutch.util.PrefixStringMatcher
Creates a new PrefixStringMatcher which will match Strings with any prefix in the supplied array.
PrefixStringMatcher(Collection) - Constructor for class org.apache.nutch.util.PrefixStringMatcher
Creates a new PrefixStringMatcher which will match Strings with any prefix in the supplied Collection.
PrefixURLFilter - Class in org.apache.nutch.urlfilter.prefix
Filters URLs based on a file of URL prefixes.
PrefixURLFilter() - Constructor for class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
PrefixURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
PrintCommandListener - Class in org.apache.nutch.protocol.ftp
This is a support class for logging all ftp command/reply traffic.
PrintCommandListener(Log) - Constructor for class org.apache.nutch.protocol.ftp.PrintCommandListener
 
processDeflateEncoded(byte[], URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
processDumpJob(String, String, Configuration, int) - Method in class org.apache.nutch.crawl.CrawlDbReader
 
processDumpJob(String, String) - Method in class org.apache.nutch.crawl.LinkDbReader
 
processGzipEncoded(byte[], URL) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
processStatJob(String, Configuration, boolean) - Method in class org.apache.nutch.crawl.CrawlDbReader
 
processTopNJob(String, long, float, String, Configuration) - Method in class org.apache.nutch.crawl.CrawlDbReader
 
PROTO_NOT_FOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
This protocol was not found.
PROTO_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
Protocol - Interface in org.apache.nutch.protocol
A retriever of url content.
PROTOCOL_REDIR - Static variable in class org.apache.nutch.fetcher.Fetcher
 
PROTOCOL_REDIR - Static variable in class org.apache.nutch.fetcher.OldFetcher
 
protocolCommandSent(ProtocolCommandEvent) - Method in class org.apache.nutch.protocol.ftp.PrintCommandListener
 
ProtocolException - Exception in org.apache.nutch.net.protocols
Deprecated. Use ProtocolException instead.
ProtocolException() - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
Deprecated.  
ProtocolException(String) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
Deprecated.  
ProtocolException(String, Throwable) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
Deprecated.  
ProtocolException(Throwable) - Constructor for exception org.apache.nutch.net.protocols.ProtocolException
Deprecated.  
ProtocolException - Exception in org.apache.nutch.protocol
 
ProtocolException() - Constructor for exception org.apache.nutch.protocol.ProtocolException
 
ProtocolException(String) - Constructor for exception org.apache.nutch.protocol.ProtocolException
 
ProtocolException(String, Throwable) - Constructor for exception org.apache.nutch.protocol.ProtocolException
 
ProtocolException(Throwable) - Constructor for exception org.apache.nutch.protocol.ProtocolException
 
ProtocolFactory - Class in org.apache.nutch.protocol
Creates and caches Protocol plugins.
ProtocolFactory(Configuration) - Constructor for class org.apache.nutch.protocol.ProtocolFactory
 
ProtocolNotFound - Exception in org.apache.nutch.protocol
 
ProtocolNotFound(String) - Constructor for exception org.apache.nutch.protocol.ProtocolNotFound
 
ProtocolNotFound(String, String) - Constructor for exception org.apache.nutch.protocol.ProtocolNotFound
 
ProtocolOutput - Class in org.apache.nutch.protocol
Simple aggregate to pass from protocol plugins both content and protocol status.
ProtocolOutput(Content, ProtocolStatus) - Constructor for class org.apache.nutch.protocol.ProtocolOutput
 
ProtocolOutput(Content) - Constructor for class org.apache.nutch.protocol.ProtocolOutput
 
protocolReplyReceived(ProtocolCommandEvent) - Method in class org.apache.nutch.protocol.ftp.PrintCommandListener
 
ProtocolStatus - Class in org.apache.nutch.protocol
 
ProtocolStatus() - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int, String[]) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int, String[], long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int, long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int, Object) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(int, Object, long) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
ProtocolStatus(Throwable) - Constructor for class org.apache.nutch.protocol.ProtocolStatus
 
proxyHost - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The proxy hostname.
proxyPort - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The proxy port.
PUBLISHER - Static variable in interface org.apache.nutch.metadata.DublinCore
An entity responsible for making the resource available.
put(Writable, Writable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
put(Text, ParseText, ParseData) - Method in class org.apache.nutch.parse.ParseResult
Store a result of parsing.
put(String, ParseText, ParseData) - Method in class org.apache.nutch.parse.ParseResult
Store a result of parsing.
putAll(MapWritable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
putAllMetaData(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
Add all metadata from other CrawlDatum to this CrawlDatum.

R

read(DataInput) - Static method in class org.apache.nutch.crawl.CrawlDatum
 
read(DataInput) - Static method in class org.apache.nutch.crawl.Inlink
 
read(DataInput) - Static method in class org.apache.nutch.parse.Outlink
 
read(DataInput) - Static method in class org.apache.nutch.parse.ParseData
 
read(DataInput) - Static method in class org.apache.nutch.parse.ParseImpl
 
read(DataInput) - Static method in class org.apache.nutch.parse.ParseStatus
 
read(DataInput) - Static method in class org.apache.nutch.parse.ParseText
 
read(DataInput) - Static method in class org.apache.nutch.protocol.Content
 
read(DataInput) - Static method in class org.apache.nutch.protocol.ProtocolStatus
 
readFields(DataInput) - Method in class org.apache.nutch.crawl.CrawlDatum
 
readFields(DataInput) - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
 
readFields(DataInput) - Method in class org.apache.nutch.crawl.Inlink
 
readFields(DataInput) - Method in class org.apache.nutch.crawl.Inlinks
 
readFields(DataInput) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
readFields(DataInput) - Method in class org.apache.nutch.fetcher.FetcherOutput
 
readFields(DataInput) - Method in class org.apache.nutch.indexer.NutchDocument
 
readFields(DataInput) - Method in class org.apache.nutch.indexer.NutchField
 
readFields(DataInput) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
readFields(DataInput) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
readFields(DataInput) - Method in class org.apache.nutch.metadata.Metadata
 
readFields(DataInput) - Method in class org.apache.nutch.metadata.MetaWrapper
 
readFields(DataInput) - Method in class org.apache.nutch.parse.Outlink
 
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseData
 
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseImpl
 
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseStatus
 
readFields(DataInput) - Method in class org.apache.nutch.parse.ParseText
 
readFields(DataInput) - Method in class org.apache.nutch.protocol.Content
 
readFields(DataInput) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
readFields(DataInput) - Method in class org.apache.nutch.scoring.webgraph.Node
 
readFields(DataInput) - Method in class org.apache.nutch.util.GenericWritableConfigurable
 
readSolrDocument(SolrDocument) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
readUrl(String, String, Configuration) - Method in class org.apache.nutch.crawl.CrawlDbReader
 
REDIR_EXCEEDED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Too many redirects.
reduce(Text, Iterator<CrawlDatum>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbMerger.Merger
 
reduce(Text, Iterator<LongWritable>, OutputCollector<Text, LongWritable>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatCombiner
 
reduce(Text, Iterator<LongWritable>, OutputCollector<Text, LongWritable>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbStatReducer
 
reduce(FloatWritable, Iterator<Text>, OutputCollector<FloatWritable, Text>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDbTopNReducer
 
reduce(Text, Iterator<CrawlDatum>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.CrawlDbReducer
 
reduce(Text, Iterator<CrawlDatum>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.Generator.CrawlDbUpdater
 
reduce(Text, Iterator<Generator.SelectorEntry>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.Generator.PartitionReducer
 
reduce(FloatWritable, Iterator<Generator.SelectorEntry>, OutputCollector<FloatWritable, Generator.SelectorEntry>, Reporter) - Method in class org.apache.nutch.crawl.Generator.Selector
Collect until limit is reached.
reduce(Text, Iterator<CrawlDatum>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.crawl.Injector.InjectReducer
 
reduce(Text, Iterator<Inlinks>, OutputCollector<Text, Inlinks>, Reporter) - Method in class org.apache.nutch.crawl.LinkDbMerger
 
reduce(Text, Iterator<NutchWritable>, OutputCollector<Text, NutchDocument>, Reporter) - Method in class org.apache.nutch.indexer.IndexerMapReduce
 
reduce(ByteWritable, Iterator<Text>, OutputCollector<Text, ByteWritable>, Reporter) - Method in class org.apache.nutch.indexer.solr.SolrClean.SolrDeleter
 
reduce(Text, Iterator<SolrDeleteDuplicates.SolrRecord>, OutputCollector<Text, SolrDeleteDuplicates.SolrRecord>, Reporter) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
reduce(Text, Iterator<Writable>, OutputCollector<Text, Writable>, Reporter) - Method in class org.apache.nutch.parse.ParseSegment
 
reduce(Text, Iterator<ObjectWritable>, OutputCollector<Text, LinkDumper.LinkNode>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Inverter
Inverts outlinks to inlinks while attaching node information to the outlink.
reduce(Text, Iterator<LinkDumper.LinkNode>, OutputCollector<Text, LinkDumper.LinkNodes>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.Merger
Aggregate all LinkNode objects for a given url.
reduce(Text, Iterator<Loops.Route>, OutputCollector<Text, Loops.LoopSet>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Finalizer
Aggregates all found routes for a given start url into a loopset and collects the loopset.
reduce(Text, Iterator<ObjectWritable>, OutputCollector<Text, Loops.Route>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Initializer
Takes any node that has inlinks and sets up a route for all of its outlinks.
reduce(Text, Iterator<ObjectWritable>, OutputCollector<Text, Loops.Route>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.Loops.Looper
Performs a single loop pass looking for loop cycles within routes.
reduce(FloatWritable, Iterator<Text>, OutputCollector<Text, FloatWritable>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper.Sorter
Flips and collects the url and numeric sort value.
reduce(Text, Iterator<ObjectWritable>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
Creates new CrawlDatum objects with the updated score from the NodeDb or with a cleared score.
reduce(Text, Iterator<LinkDatum>, OutputCollector<Text, LinkDatum>, Reporter) - Method in class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
 
reduce(Text, Iterator<MetaWrapper>, OutputCollector<Text, MetaWrapper>, Reporter) - Method in class org.apache.nutch.segment.SegmentMerger
NOTE: in selecting the latest version we rely exclusively on the segment name (not all segment data contain time information).
reduce(Text, Iterator<NutchWritable>, OutputCollector<Text, Text>, Reporter) - Method in class org.apache.nutch.segment.SegmentReader
 
reduce(Text, Iterator<CrawlDatum>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.tools.CrawlDBScanner
 
reduce(Text, Iterator<Generator.SelectorEntry>, OutputCollector<Text, CrawlDatum>, Reporter) - Method in class org.apache.nutch.tools.FreeGenerator.FG
 
reduce(Text, Iterator<LongWritable>, OutputCollector<Text, LongWritable>, Reporter) - Method in class org.apache.nutch.util.domain.DomainStatistics.DomainStatisticsCombiner
 
reduce(Text, Iterator<LongWritable>, OutputCollector<LongWritable, Text>, Reporter) - Method in class org.apache.nutch.util.domain.DomainStatistics
 
RegexRule - Class in org.apache.nutch.urlfilter.api
A generic regular expression rule.
RegexRule(boolean, String) - Constructor for class org.apache.nutch.urlfilter.api.RegexRule
Constructs a new regular expression rule.
RegexURLFilter - Class in org.apache.nutch.urlfilter.regex
Filters URLs based on a file of regular expressions using the Java Regex implementation.
RegexURLFilter() - Constructor for class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
RegexURLFilter(String) - Constructor for class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
RegexURLFilterBase - Class in org.apache.nutch.urlfilter.api
Generic URL filter based on regular expressions.
RegexURLFilterBase() - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Constructs a new empty RegexURLFilterBase
RegexURLFilterBase(File) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Constructs a new RegexURLFilter and init it with a file of rules.
RegexURLFilterBase(String) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Constructs a new RegexURLFilter and inits it with a list of rules.
RegexURLFilterBase(Reader) - Constructor for class org.apache.nutch.urlfilter.api.RegexURLFilterBase
Constructs a new RegexURLFilter and init it with a Reader of rules.
REL_TAG - Static variable in class org.apache.nutch.microformats.reltag.RelTagParser
 
RELATION - Static variable in interface org.apache.nutch.metadata.DublinCore
A reference to a related resource.
RelTagIndexingFilter - Class in org.apache.nutch.microformats.reltag
An IndexingFilter that add tag field(s) to the document.
RelTagIndexingFilter() - Constructor for class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
 
RelTagParser - Class in org.apache.nutch.microformats.reltag
Adds microformat rel-tags of document if found.
RelTagParser() - Constructor for class org.apache.nutch.microformats.reltag.RelTagParser
 
remove(Writable) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
remove(String) - Method in class org.apache.nutch.metadata.Metadata
Remove a metadata and all its associated values.
remove(String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
 
removeField(String) - Method in class org.apache.nutch.indexer.NutchDocument
 
removeLockFile(FileSystem, Path) - Static method in class org.apache.nutch.util.LockUtil
Remove lock file.
replace(FileSystem, Path, Path, boolean) - Static method in class org.apache.nutch.util.FSUtils
Replaces the current path with the new path and if set removes the old path.
REPR_URL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
reset() - Method in class org.apache.nutch.indexer.NutchField
 
reset() - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets all boolean values to false.
resolveEncodingAlias(String) - Static method in class org.apache.nutch.util.EncodingDetector
 
ResolveUrls - Class in org.apache.nutch.tools
A simple tool that will spin up multiple threads to resolve urls to ip addresses.
ResolveUrls(String) - Constructor for class org.apache.nutch.tools.ResolveUrls
Create a new ResolveUrls with a file from the local file system.
ResolveUrls(String, int) - Constructor for class org.apache.nutch.tools.ResolveUrls
Create a new ResolveUrls with a urls file and a number of threads for the Thread pool.
resolveUrls() - Method in class org.apache.nutch.tools.ResolveUrls
Creates a thread pool for resolving urls.
Response - Interface in org.apache.nutch.net.protocols
A response inteface.
retrieveFile(String, OutputStream, int) - Method in class org.apache.nutch.protocol.ftp.Client
 
retrieveList(String, List, int, FTPFileEntryParser) - Method in class org.apache.nutch.protocol.ftp.Client
 
RETRY - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Temporary failure.
REVISION_NUMBER - Static variable in interface org.apache.nutch.metadata.Office
 
rightPad(String, int) - Static method in class org.apache.nutch.util.StringUtil
Returns a copy of s padded with trailing spaces so that it's length is length.
RIGHTS - Static variable in interface org.apache.nutch.metadata.DublinCore
Information about rights held in and over the resource.
RobotRules - Interface in org.apache.nutch.protocol
This class holds the rules which were parsed from a robots.txt file, and can test paths against those rules.
RobotRulesParser - Class in org.apache.nutch.protocol.http.api
This class handles the parsing of robots.txt files.
RobotRulesParser(Configuration) - Constructor for class org.apache.nutch.protocol.http.api.RobotRulesParser
 
RobotRulesParser.RobotRuleSet - Class in org.apache.nutch.protocol.http.api
This class holds the rules which were parsed from a robots.txt file, and can test paths against those rules.
RobotRulesParser.RobotRuleSet() - Constructor for class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
 
ROBOTS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Access denied by robots.txt rules.
root - Variable in class org.apache.nutch.util.TrieStringMatcher
 
ROUTES_DIR - Static variable in class org.apache.nutch.scoring.webgraph.Loops
 
RULES - Static variable in class org.apache.nutch.protocol.EmptyRobotRules
 
run(String[]) - Method in class org.apache.nutch.crawl.Crawl
 
run(String[]) - Method in class org.apache.nutch.crawl.CrawlDb
 
run(String[]) - Method in class org.apache.nutch.crawl.CrawlDbMerger
 
run(String[]) - Method in class org.apache.nutch.crawl.Generator
 
run(String[]) - Method in class org.apache.nutch.crawl.Injector
 
run(String[]) - Method in class org.apache.nutch.crawl.LinkDb
 
run(String[]) - Method in class org.apache.nutch.crawl.LinkDbMerger
 
run(String[]) - Method in class org.apache.nutch.crawl.LinkDbReader
 
run(RecordReader<Text, CrawlDatum>, OutputCollector<Text, NutchWritable>, Reporter) - Method in class org.apache.nutch.fetcher.Fetcher
 
run(String[]) - Method in class org.apache.nutch.fetcher.Fetcher
 
run(RecordReader<WritableComparable, Writable>, OutputCollector<Text, NutchWritable>, Reporter) - Method in class org.apache.nutch.fetcher.OldFetcher
 
run(String[]) - Method in class org.apache.nutch.fetcher.OldFetcher
 
run(String[]) - Method in class org.apache.nutch.indexer.solr.SolrClean
 
run(String[]) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
run(String[]) - Method in class org.apache.nutch.indexer.solr.SolrIndexer
 
run(String[]) - Method in class org.apache.nutch.parse.ParseSegment
 
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper
Runs the LinkDumper tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.LinkRank
Runs the LinkRank tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.Loops
Runs the Loops tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.NodeDumper
Runs the node dumper tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
Runs the ScoreUpdater tool.
run(String[]) - Method in class org.apache.nutch.scoring.webgraph.WebGraph
Parses command link arguments and runs the WebGraph jobs.
run(String[]) - Method in class org.apache.nutch.tools.arc.ArcSegmentCreator
 
run(String[]) - Method in class org.apache.nutch.tools.Benchmark
 
run(String[]) - Method in class org.apache.nutch.tools.CrawlDBScanner
 
run(String[]) - Method in class org.apache.nutch.tools.FreeGenerator
 
run(String[]) - Method in class org.apache.nutch.util.domain.DomainStatistics
 

S

save(OutputStream) - Method in class org.apache.nutch.analysis.lang.NGramProfile
Writes NGramProfile content into OutputStream, content is outputted with UTF-8 encoding
saveDom(OutputStream, Element) - Static method in class org.apache.nutch.util.DomUtil
save dom into ouputstream
SCOPE_CRAWLDB - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used when updating the CrawlDb with new URLs.
SCOPE_DEFAULT - Static variable in class org.apache.nutch.net.URLNormalizers
Default scope.
SCOPE_FETCHER - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used by Fetcher when processing redirect URLs.
SCOPE_GENERATE_HOST_COUNT - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used by Generator.
SCOPE_INJECT - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used by Injector.
SCOPE_LINKDB - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used when updating the LinkDb with new URLs.
SCOPE_OUTLINK - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used when constructing new Outlink instances.
SCOPE_PARTITION - Static variable in class org.apache.nutch.net.URLNormalizers
Scope used by URLPartitioner.
SCORE_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
ScoreUpdater - Class in org.apache.nutch.scoring.webgraph
Updates the score from the WebGraph node database into the crawl database.
ScoreUpdater() - Constructor for class org.apache.nutch.scoring.webgraph.ScoreUpdater
 
ScoringFilter - Interface in org.apache.nutch.scoring
A contract defining behavior of scoring plugins.
ScoringFilterException - Exception in org.apache.nutch.scoring
Specialized exception for errors during scoring.
ScoringFilterException() - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
 
ScoringFilterException(String) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
 
ScoringFilterException(String, Throwable) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
 
ScoringFilterException(Throwable) - Constructor for exception org.apache.nutch.scoring.ScoringFilterException
 
ScoringFilters - Class in org.apache.nutch.scoring
Creates and caches ScoringFilter implementing plugins.
ScoringFilters(Configuration) - Constructor for class org.apache.nutch.scoring.ScoringFilters
 
SECONDS_PER_DAY - Static variable in interface org.apache.nutch.crawl.FetchSchedule
 
SEGMENT_NAME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
SegmentHandler - Class in org.apache.nutch.tools.proxy
XXX should turn this into a plugin?
SegmentHandler(Configuration, Path) - Constructor for class org.apache.nutch.tools.proxy.SegmentHandler
 
SegmentMergeFilter - Interface in org.apache.nutch.segment
Interface used to filter segments during segment merge.
SegmentMergeFilters - Class in org.apache.nutch.segment
This class wraps all SegmentMergeFilter extensions in a single object so it is easier to operate on them.
SegmentMergeFilters(Configuration) - Constructor for class org.apache.nutch.segment.SegmentMergeFilters
 
SegmentMerger - Class in org.apache.nutch.segment
This tool takes several segments and merges their data together.
SegmentMerger() - Constructor for class org.apache.nutch.segment.SegmentMerger
 
SegmentMerger(Configuration) - Constructor for class org.apache.nutch.segment.SegmentMerger
 
SegmentMerger.ObjectInputFormat - Class in org.apache.nutch.segment
Wraps inputs in an MetaWrapper, to permit merging different types in reduce and use additional metadata.
SegmentMerger.ObjectInputFormat() - Constructor for class org.apache.nutch.segment.SegmentMerger.ObjectInputFormat
 
SegmentMerger.SegmentOutputFormat - Class in org.apache.nutch.segment
 
SegmentMerger.SegmentOutputFormat() - Constructor for class org.apache.nutch.segment.SegmentMerger.SegmentOutputFormat
 
segmentName - Variable in class org.apache.nutch.segment.SegmentPart
Name of the segment (just the last path component).
SegmentPart - Class in org.apache.nutch.segment
Utility class for handling information about segment parts.
SegmentPart() - Constructor for class org.apache.nutch.segment.SegmentPart
 
SegmentPart(String, String) - Constructor for class org.apache.nutch.segment.SegmentPart
 
SegmentReader - Class in org.apache.nutch.segment
Dump the content of a segment.
SegmentReader() - Constructor for class org.apache.nutch.segment.SegmentReader
 
SegmentReader(Configuration, boolean, boolean, boolean, boolean, boolean, boolean) - Constructor for class org.apache.nutch.segment.SegmentReader
 
SegmentReader.InputCompatMapper - Class in org.apache.nutch.segment
 
SegmentReader.InputCompatMapper() - Constructor for class org.apache.nutch.segment.SegmentReader.InputCompatMapper
 
SegmentReader.SegmentReaderStats - Class in org.apache.nutch.segment
 
SegmentReader.SegmentReaderStats() - Constructor for class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
SegmentReader.TextOutputFormat - Class in org.apache.nutch.segment
Implements a text output format
SegmentReader.TextOutputFormat() - Constructor for class org.apache.nutch.segment.SegmentReader.TextOutputFormat
 
segnum - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
 
sendNoOp() - Method in class org.apache.nutch.protocol.ftp.Client
Sends a NOOP command to the FTP server.
SERVER_URL - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
set(CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDatum
Copy the contents of another instance into this instance.
set(String, String) - Method in class org.apache.nutch.metadata.Metadata
Set metadata name/value.
set(String, String) - Method in class org.apache.nutch.metadata.SpellCheckedMetadata
 
setAll(Properties) - Method in class org.apache.nutch.metadata.Metadata
Copy All key-value pairs from properties.
setAnchor(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
setArgs(String[]) - Method in class org.apache.nutch.parse.ParseStatus
 
setArgs(String[]) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
setBaseHref(URL) - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets the baseHref.
setClazz(String) - Method in class org.apache.nutch.plugin.Extension
Sets the Class that implement the concret extension and is only used until model creation at system start up.
setCode(int) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
setCommand(String) - Method in class org.apache.nutch.util.CommandRunner
 
setConf(Configuration) - Method in class org.apache.nutch.analysis.lang.HTMLLanguageParser
 
setConf(Configuration) - Method in class org.apache.nutch.analysis.lang.LanguageIndexingFilter
 
setConf(Configuration) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
 
setConf(Configuration) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
 
setConf(Configuration) - Method in class org.apache.nutch.crawl.Signature
 
setConf(Configuration) - Method in class org.apache.nutch.indexer.basic.BasicIndexingFilter
 
setConf(Configuration) - Method in class org.apache.nutch.indexer.more.MoreIndexingFilter
 
setConf(Configuration) - Method in class org.apache.nutch.indexer.solr.SolrClean
 
setConf(Configuration) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
setConf(Configuration) - Method in class org.apache.nutch.microformats.reltag.RelTagIndexingFilter
 
setConf(Configuration) - Method in class org.apache.nutch.microformats.reltag.RelTagParser
 
setConf(Configuration) - Method in class org.apache.nutch.parse.ext.ExtParser
 
setConf(Configuration) - Method in class org.apache.nutch.parse.js.JSParseFilter
 
setConf(Configuration) - Method in class org.apache.nutch.parse.swf.SWFParser
 
setConf(Configuration) - Method in class org.apache.nutch.parse.tika.TikaParser
 
setConf(Configuration) - Method in class org.apache.nutch.parse.zip.ZipParser
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.file.File
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.ftp.Ftp
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.http.Http
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.Http
Reads the configuration from the Nutch configuration files and sets the configuration.
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
 
setConf(Configuration) - Method in class org.apache.nutch.protocol.httpclient.HttpBasicAuthentication
 
setConf(Configuration) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
 
setConf(Configuration) - Method in class org.apache.nutch.segment.SegmentMerger
 
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.api.RegexURLFilterBase
 
setConf(Configuration) - Method in class org.apache.nutch.urlfilter.prefix.PrefixURLFilter
 
setConf(Configuration) - Method in class org.apache.nutch.util.domain.DomainStatistics
 
setConf(Configuration) - Method in class org.apache.nutch.util.GenericWritableConfigurable
 
setConf(Configuration) - Method in class org.creativecommons.nutch.CCIndexingFilter
 
setConf(Configuration) - Method in class org.creativecommons.nutch.CCParseFilter
 
setContent(byte[]) - Method in class org.apache.nutch.protocol.Content
 
setContent(Content) - Method in class org.apache.nutch.protocol.ProtocolOutput
 
setContentType(String) - Method in class org.apache.nutch.protocol.Content
 
setCrawlDelay(long) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Set Crawl-Delay, in milliseconds
setDataTimeout(int) - Method in class org.apache.nutch.protocol.ftp.Client
Sets the timeout in milliseconds to use for data connection.
setDescriptor(PluginDescriptor) - Method in class org.apache.nutch.plugin.Extension
Sets the plugin descriptor and is only used until model creation at system start up.
setExpireTime(long) - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
Change when the ruleset goes stale.
setFetchInterval(int) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setFetchInterval(float) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.AdaptiveFetchSchedule
 
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in class org.apache.nutch.crawl.DefaultFetchSchedule
 
setFetchSchedule(Text, CrawlDatum, long, long, long, long, int) - Method in interface org.apache.nutch.crawl.FetchSchedule
Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchTime(long) - Method in class org.apache.nutch.crawl.CrawlDatum
Sets either the time of the last fetch or the next fetch time, depending on whether Fetcher or CrawlDbReducer set the time.
setFileType(int) - Method in class org.apache.nutch.protocol.ftp.Client
Sets the file type to be transferred.
setFollowTalk(boolean) - Method in class org.apache.nutch.protocol.ftp.Ftp
Set followTalk
setFound(boolean) - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
setId(String) - Method in class org.apache.nutch.plugin.Extension
Sets the unique extension Id and is only used until model creation at system start up.
setInlinkScore(float) - Method in class org.apache.nutch.scoring.webgraph.Node
 
setInputStream(InputStream) - Method in class org.apache.nutch.util.CommandRunner
 
setKeepConnection(boolean) - Method in class org.apache.nutch.protocol.ftp.Ftp
Set keepConnection
setLastModified(long) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
setLinks(LinkDumper.LinkNode[]) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
setLinkType(byte) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
setLookingFor(String) - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
setLoopSet(Set<String>) - Method in class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
setMajorCode(byte) - Method in class org.apache.nutch.parse.ParseStatus
 
setMaxContentLength(int) - Method in class org.apache.nutch.protocol.file.File
Set the point at which content is truncated.
setMaxContentLength(int) - Method in class org.apache.nutch.protocol.ftp.Ftp
Set the point at which content is truncated.
setMessage(String) - Method in class org.apache.nutch.parse.ParseStatus
 
setMessage(String) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
setMeta(String, String) - Method in class org.apache.nutch.metadata.MetaWrapper
Set metadata.
setMetaData(MapWritable) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setMetadata(Metadata) - Method in class org.apache.nutch.protocol.Content
Other protocol-specific data.
setMetadata(Metadata) - Method in class org.apache.nutch.scoring.webgraph.Node
 
setMinorCode(short) - Method in class org.apache.nutch.parse.ParseStatus
 
setModifiedTime(long) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setNoCache() - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets noCache to true.
setNode(Node) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
setNoFollow() - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets noFollow to true.
setNoIndex() - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets noIndex to true.
setNumInlinks(int) - Method in class org.apache.nutch.scoring.webgraph.Node
 
setNumOutlinks(int) - Method in class org.apache.nutch.scoring.webgraph.Node
 
setObject(String, Object) - Method in class org.apache.nutch.util.ObjectCache
 
setOutlinkUrl(String) - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
setPageGoneSchedule(Text, CrawlDatum, long, long, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(Text, CrawlDatum, long, long, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
This method specifies how to schedule refetching of pages marked as GONE.
setPageRetrySchedule(Text, CrawlDatum, long, long, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setPageRetrySchedule(Text, CrawlDatum, long, long, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setParseMeta(Metadata) - Method in class org.apache.nutch.parse.ParseData
 
setRefresh(boolean) - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets refresh to the supplied value.
setRefreshHref(URL) - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets the refreshHref.
setRefreshTime(int) - Method in class org.apache.nutch.parse.HTMLMetaTags
Sets the refreshTime.
setRemoteVerificationEnabled(boolean) - Method in class org.apache.nutch.protocol.ftp.Client
Enable or disable verification that the remote host taking part of a data connection is the same as the host to which the control connection is attached.
setRetriesSinceFetch(int) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setScore(float) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setScore(float) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
setSignature(byte[]) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setStatus(int) - Method in class org.apache.nutch.crawl.CrawlDatum
 
setStatus(ProtocolStatus) - Method in class org.apache.nutch.protocol.ProtocolOutput
 
setStdErrorStream(OutputStream) - Method in class org.apache.nutch.util.CommandRunner
 
setStdOutputStream(OutputStream) - Method in class org.apache.nutch.util.CommandRunner
 
setTimeout(int) - Method in class org.apache.nutch.protocol.ftp.Ftp
Set the timeout.
setTimeout(int) - Method in class org.apache.nutch.util.CommandRunner
 
setTimestamp(long) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
setUrl(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
setUrl(String) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
setWaitForExit(boolean) - Method in class org.apache.nutch.util.CommandRunner
 
setWeight(float) - Method in class org.apache.nutch.indexer.NutchDocument
 
setWeight(float) - Method in class org.apache.nutch.indexer.NutchField
 
shortestMatch(String) - Method in class org.apache.nutch.util.PrefixStringMatcher
Returns the shortest prefix of input that is matched, or null if no match exists.
shortestMatch(String) - Method in class org.apache.nutch.util.SuffixStringMatcher
Returns the shortest suffix of input that is matched, or null if no match exists.
shortestMatch(String) - Method in class org.apache.nutch.util.TrieStringMatcher
Returns the shortest substring of input that is matched by a pattern in the trie, or null if no match exists.
shouldFetch(Text, CrawlDatum, long) - Method in class org.apache.nutch.crawl.AbstractFetchSchedule
This method provides information whether the page is suitable for selection in the current fetchlist.
shouldFetch(Text, CrawlDatum, long) - Method in interface org.apache.nutch.crawl.FetchSchedule
This method provides information whether the page is suitable for selection in the current fetchlist.
shutDown() - Method in class org.apache.nutch.plugin.Plugin
Shutdown the plugin.
Signature - Class in org.apache.nutch.crawl
 
Signature() - Constructor for class org.apache.nutch.crawl.Signature
 
SIGNATURE_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
SignatureComparator - Class in org.apache.nutch.crawl
 
SignatureComparator() - Constructor for class org.apache.nutch.crawl.SignatureComparator
 
SignatureFactory - Class in org.apache.nutch.crawl
Factory class, which instantiates a Signature implementation according to the current Configuration configuration.
size() - Method in class org.apache.nutch.crawl.Inlinks
 
size() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
size() - Method in class org.apache.nutch.metadata.Metadata
Returns the number of metadata names in this metadata.
size() - Method in class org.apache.nutch.parse.ParseResult
Return the number of parse outputs (both successful and failed)
skip(DataInput) - Static method in class org.apache.nutch.crawl.Inlink
Skips over one Inlink in the input.
skip(DataInput) - Static method in class org.apache.nutch.parse.Outlink
Skips over one Outlink in the input.
skipChildren() - Method in class org.apache.nutch.util.NodeWalker
Skips over and removes from the node stack the children of the last node.
SOLR_PREFIX - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
SolrClean - Class in org.apache.nutch.indexer.solr
The class scans CrawlDB looking for entries with status DB_GONE (404) and sends delete requests to Solr for those documents.
SolrClean() - Constructor for class org.apache.nutch.indexer.solr.SolrClean
 
SolrClean.DBFilter - Class in org.apache.nutch.indexer.solr
 
SolrClean.DBFilter() - Constructor for class org.apache.nutch.indexer.solr.SolrClean.DBFilter
 
SolrClean.SolrDeleter - Class in org.apache.nutch.indexer.solr
 
SolrClean.SolrDeleter() - Constructor for class org.apache.nutch.indexer.solr.SolrClean.SolrDeleter
 
SolrConstants - Interface in org.apache.nutch.indexer.solr
 
SolrDeleteDuplicates - Class in org.apache.nutch.indexer.solr
Utility class for deleting duplicate documents from a solr index.
SolrDeleteDuplicates() - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates
 
SolrDeleteDuplicates.SolrInputFormat - Class in org.apache.nutch.indexer.solr
 
SolrDeleteDuplicates.SolrInputFormat() - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputFormat
 
SolrDeleteDuplicates.SolrInputSplit - Class in org.apache.nutch.indexer.solr
 
SolrDeleteDuplicates.SolrInputSplit() - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
SolrDeleteDuplicates.SolrInputSplit(int, int) - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
SolrDeleteDuplicates.SolrRecord - Class in org.apache.nutch.indexer.solr
 
SolrDeleteDuplicates.SolrRecord() - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
SolrDeleteDuplicates.SolrRecord(SolrDeleteDuplicates.SolrRecord) - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
SolrDeleteDuplicates.SolrRecord(String, float, long) - Constructor for class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
SolrIndexer - Class in org.apache.nutch.indexer.solr
 
SolrIndexer() - Constructor for class org.apache.nutch.indexer.solr.SolrIndexer
 
SolrIndexer(Configuration) - Constructor for class org.apache.nutch.indexer.solr.SolrIndexer
 
SolrMappingReader - Class in org.apache.nutch.indexer.solr
 
SolrMappingReader(Configuration) - Constructor for class org.apache.nutch.indexer.solr.SolrMappingReader
 
SolrWriter - Class in org.apache.nutch.indexer.solr
 
SolrWriter() - Constructor for class org.apache.nutch.indexer.solr.SolrWriter
 
SOURCE - Static variable in interface org.apache.nutch.metadata.DublinCore
A reference to a resource from which the present resource is derived.
SpellCheckedMetadata - Class in org.apache.nutch.metadata
A decorator to Metadata that adds spellchecking capabilities to property names.
SpellCheckedMetadata() - Constructor for class org.apache.nutch.metadata.SpellCheckedMetadata
 
splitEnd - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
splitLen - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
splitStart - Variable in class org.apache.nutch.tools.arc.ArcRecordReader
 
start - Variable in class org.apache.nutch.segment.SegmentReader.SegmentReaderStats
 
startUp() - Method in class org.apache.nutch.plugin.Plugin
Will be invoked until plugin start up.
statNames - Static variable in class org.apache.nutch.crawl.CrawlDatum
 
STATUS_BLOCKED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_DB_FETCHED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page was successfully fetched.
STATUS_DB_GONE - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page no longer exists.
STATUS_DB_MAX - Static variable in class org.apache.nutch.crawl.CrawlDatum
Maximum value of DB-related status.
STATUS_DB_NOTMODIFIED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page was successfully fetched and found not modified.
STATUS_DB_REDIR_PERM - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page permanently redirects to other page.
STATUS_DB_REDIR_TEMP - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page temporarily redirects to other page.
STATUS_DB_UNFETCHED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page was not fetched yet.
STATUS_FAILED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_FAILURE - Static variable in class org.apache.nutch.parse.ParseStatus
 
STATUS_FETCH_GONE - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching unsuccessful - page is gone.
STATUS_FETCH_MAX - Static variable in class org.apache.nutch.crawl.CrawlDatum
Maximum value of fetch-related status.
STATUS_FETCH_NOTMODIFIED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching successful - page is not modified.
STATUS_FETCH_REDIR_PERM - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching permanently redirected to other page.
STATUS_FETCH_REDIR_TEMP - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching temporarily redirected to other page.
STATUS_FETCH_RETRY - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching unsuccessful, needs to be retried (transient errors).
STATUS_FETCH_SUCCESS - Static variable in class org.apache.nutch.crawl.CrawlDatum
Fetching was successful.
STATUS_GONE - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_INJECTED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page was newly injected.
STATUS_LINKED - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page discovered through a link.
STATUS_MODIFIED - Static variable in interface org.apache.nutch.crawl.FetchSchedule
Page is known to have been modified since our last visit.
STATUS_NOTFETCHING - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_NOTFOUND - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_NOTMODIFIED - Static variable in interface org.apache.nutch.crawl.FetchSchedule
Page is known to remain unmodified since our last visit.
STATUS_NOTMODIFIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_NOTPARSED - Static variable in class org.apache.nutch.parse.ParseStatus
 
STATUS_PARSE_META - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page got metadata from a parser
STATUS_REDIR_EXCEEDED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_RETRY - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_ROBOTS_DENIED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_SIGNATURE - Static variable in class org.apache.nutch.crawl.CrawlDatum
Page signature.
STATUS_SUCCESS - Static variable in class org.apache.nutch.parse.ParseStatus
 
STATUS_SUCCESS - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STATUS_UNKNOWN - Static variable in interface org.apache.nutch.crawl.FetchSchedule
It is unknown whether page was changed since our last visit.
STATUS_WOULDBLOCK - Static variable in class org.apache.nutch.protocol.ProtocolStatus
 
STD_FORMAT - Static variable in class org.apache.nutch.crawl.CrawlDbReader
 
StringUtil - Class in org.apache.nutch.util
A collection of String processing utility methods.
StringUtil() - Constructor for class org.apache.nutch.util.StringUtil
 
SUBJECT - Static variable in interface org.apache.nutch.metadata.DublinCore
The topic of the content of the resource.
SUCCESS - Static variable in class org.apache.nutch.parse.ParseStatus
Parsing succeeded.
SUCCESS - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Content was retrieved without errors.
SUCCESS_REDIRECT - Static variable in class org.apache.nutch.parse.ParseStatus
Parsed content contains a directive to redirect to another URL.
SuffixStringMatcher - Class in org.apache.nutch.util
A class for efficiently matching Strings against a set of suffixes.
SuffixStringMatcher(String[]) - Constructor for class org.apache.nutch.util.SuffixStringMatcher
Creates a new PrefixStringMatcher which will match Strings with any suffix in the supplied array.
SuffixStringMatcher(Collection) - Constructor for class org.apache.nutch.util.SuffixStringMatcher
Creates a new PrefixStringMatcher which will match Strings with any suffix in the supplied Collection
SWFParser - Class in org.apache.nutch.parse.swf
Parser for Flash SWF files.
SWFParser() - Constructor for class org.apache.nutch.parse.swf.SWFParser
 

T

TEMP_MOVED - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Resource has moved temporarily.
TEMPLATE - Static variable in interface org.apache.nutch.metadata.Office
 
terminal - Variable in class org.apache.nutch.util.TrieStringMatcher.TrieNode
 
TestbedProxy - Class in org.apache.nutch.tools.proxy
 
TestbedProxy() - Constructor for class org.apache.nutch.tools.proxy.TestbedProxy
 
TextProfileSignature - Class in org.apache.nutch.crawl
An implementation of a page signature.
TextProfileSignature() - Constructor for class org.apache.nutch.crawl.TextProfileSignature
 
TikaConfig - Class in org.apache.nutch.parse.tika
Parse xml config file.
TikaConfig(String) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(File) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(URL) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(InputStream) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(InputStream, Parser) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
Deprecated. This method will be removed in Apache Tika 1.0
TikaConfig(Document) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(Document, Parser) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
Deprecated. This method will be removed in Apache Tika 1.0
TikaConfig(Element) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig() - Constructor for class org.apache.nutch.parse.tika.TikaConfig
 
TikaConfig(Element, Parser) - Constructor for class org.apache.nutch.parse.tika.TikaConfig
Deprecated. This method will be removed in Apache Tika 1.0
TikaParser - Class in org.apache.nutch.parse.tika
Wrapper for Tika parsers.
TikaParser() - Constructor for class org.apache.nutch.parse.tika.TikaParser
 
timeout - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The network timeout in millisecond
TIMESTAMP_FIELD - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
TimingUtil - Class in org.apache.nutch.util
 
TimingUtil() - Constructor for class org.apache.nutch.util.TimingUtil
 
TITLE - Static variable in interface org.apache.nutch.metadata.DublinCore
A name given to the resource.
toContent() - Method in class org.apache.nutch.protocol.file.FileResponse
 
toContent() - Method in class org.apache.nutch.protocol.ftp.FtpResponse
 
toDate(String) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
 
toHexString(byte[]) - Static method in class org.apache.nutch.util.StringUtil
Convenience call for StringUtil.toHexString(byte[], String, int), where sep = null; lineLen = Integer.MAX_VALUE.
toHexString(byte[], String, int) - Static method in class org.apache.nutch.util.StringUtil
Get a text representation of a byte[] as hexadecimal String, where each pair of hexadecimal digits corresponds to consecutive bytes in the array.
toLong(String) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
 
TopLevelDomain - Class in org.apache.nutch.util.domain
(From wikipedia) A top-level domain (TLD) is the last part of an Internet domain name; that is, the letters which follow the final dot of any domain name.
TopLevelDomain(String, TopLevelDomain.Type, DomainSuffix.Status, float) - Constructor for class org.apache.nutch.util.domain.TopLevelDomain
 
TopLevelDomain(String, DomainSuffix.Status, float, String) - Constructor for class org.apache.nutch.util.domain.TopLevelDomain
 
TopLevelDomain.Type - Enum in org.apache.nutch.util.domain
 
toString() - Method in class org.apache.nutch.analysis.lang.NGramProfile
 
toString() - Method in class org.apache.nutch.crawl.CrawlDatum
 
toString() - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
 
toString() - Method in class org.apache.nutch.crawl.Inlink
 
toString() - Method in class org.apache.nutch.crawl.Inlinks
 
toString() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
toString() - Method in class org.apache.nutch.fetcher.FetcherOutput
 
toString() - Method in class org.apache.nutch.metadata.Metadata
 
toString(Date) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
Get the HTTP format of the specified date.
toString(Calendar) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
 
toString(long) - Static method in class org.apache.nutch.net.protocols.HttpDateFormat
 
toString() - Method in class org.apache.nutch.parse.HTMLMetaTags
 
toString() - Method in class org.apache.nutch.parse.Outlink
 
toString() - Method in class org.apache.nutch.parse.ParseData
 
toString() - Method in class org.apache.nutch.parse.ParseStatus
 
toString() - Method in class org.apache.nutch.parse.ParseText
 
toString() - Method in class org.apache.nutch.protocol.Content
 
toString() - Method in class org.apache.nutch.protocol.http.api.RobotRulesParser.RobotRuleSet
 
toString() - Method in class org.apache.nutch.protocol.ProtocolStatus
 
toString() - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
toString() - Method in class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
toString() - Method in class org.apache.nutch.scoring.webgraph.Node
 
toString() - Method in class org.apache.nutch.segment.SegmentPart
Return a String representation of this class, in the form "segmentName/partName".
toString() - Method in class org.apache.nutch.tools.Benchmark.BenchmarkResults
 
toString() - Method in class org.apache.nutch.util.domain.DomainSuffix
 
TrieStringMatcher - Class in org.apache.nutch.util
TrieStringMatcher is a base class for simple tree-based string matching.
TrieStringMatcher() - Constructor for class org.apache.nutch.util.TrieStringMatcher
 
TrieStringMatcher.TrieNode - Class in org.apache.nutch.util
Node class for the character tree.
TYPE - Static variable in interface org.apache.nutch.metadata.DublinCore
The nature or genre of the content of the resource.

U

unzip(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
Returns an gunzipped copy of the input array.
unzipBestEffort(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
Returns an gunzipped copy of the input array.
unzipBestEffort(byte[], int) - Static method in class org.apache.nutch.util.GZIPUtils
Returns an gunzipped copy of the input array, truncated to sizeLimit bytes, if necessary.
update(Path, Path[], boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDb
 
update(Path, Path[], boolean, boolean, boolean, boolean) - Method in class org.apache.nutch.crawl.CrawlDb
 
update(Path, Path) - Method in class org.apache.nutch.scoring.webgraph.ScoreUpdater
Updates the inlink score in the web graph node databsae into the crawl database.
updateDbScore(Text, CrawlDatum, CrawlDatum, List) - Method in class org.apache.nutch.scoring.opic.OPICScoringFilter
Increase the score by a sum of inlinked scores.
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in interface org.apache.nutch.scoring.ScoringFilter
This method calculates a new score of CrawlDatum during CrawlDb update, based on the initial value of the original CrawlDatum, and also score values contributed by inlinked pages.
updateDbScore(Text, CrawlDatum, CrawlDatum, List<CrawlDatum>) - Method in class org.apache.nutch.scoring.ScoringFilters
Calculate updated page score during CrawlDb.update().
url - Variable in class org.apache.nutch.crawl.Generator.SelectorEntry
 
URL_FIELD - Static variable in interface org.apache.nutch.indexer.solr.SolrConstants
 
URL_FILTERING - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
 
URL_FILTERING - Static variable in class org.apache.nutch.crawl.LinkDbFilter
 
URL_NORMALIZING - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
 
URL_NORMALIZING - Static variable in class org.apache.nutch.crawl.LinkDbFilter
 
URL_NORMALIZING_SCOPE - Static variable in class org.apache.nutch.crawl.CrawlDbFilter
 
URL_NORMALIZING_SCOPE - Static variable in class org.apache.nutch.crawl.LinkDbFilter
 
URL_VERSION - Static variable in class org.apache.nutch.tools.arc.ArcSegmentCreator
 
URLFilter - Interface in org.apache.nutch.net
Interface used to limit which URLs enter Nutch.
URLFILTER_AUTOMATON_FILE - Static variable in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
URLFILTER_AUTOMATON_RULES - Static variable in class org.apache.nutch.urlfilter.automaton.AutomatonURLFilter
 
URLFILTER_ORDER - Static variable in class org.apache.nutch.net.URLFilters
 
URLFILTER_REGEX_FILE - Static variable in class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
URLFILTER_REGEX_RULES - Static variable in class org.apache.nutch.urlfilter.regex.RegexURLFilter
 
URLFilterChecker - Class in org.apache.nutch.net
Checks one given filter or all filters.
URLFilterChecker(Configuration) - Constructor for class org.apache.nutch.net.URLFilterChecker
 
URLFilterException - Exception in org.apache.nutch.net
 
URLFilterException() - Constructor for exception org.apache.nutch.net.URLFilterException
 
URLFilterException(String) - Constructor for exception org.apache.nutch.net.URLFilterException
 
URLFilterException(String, Throwable) - Constructor for exception org.apache.nutch.net.URLFilterException
 
URLFilterException(Throwable) - Constructor for exception org.apache.nutch.net.URLFilterException
 
URLFilters - Class in org.apache.nutch.net
Creates and caches URLFilter implementing plugins.
URLFilters(Configuration) - Constructor for class org.apache.nutch.net.URLFilters
 
URLNormalizer - Interface in org.apache.nutch.net
Interface used to convert URLs to normal form and optionally perform substitutions
URLNormalizerChecker - Class in org.apache.nutch.net
Checks one given normalizer or all normalizers.
URLNormalizerChecker(Configuration) - Constructor for class org.apache.nutch.net.URLNormalizerChecker
 
URLNormalizers - Class in org.apache.nutch.net
This class uses a "chained filter" pattern to run defined normalizers.
URLNormalizers(Configuration, String) - Constructor for class org.apache.nutch.net.URLNormalizers
 
URLPartitioner - Class in org.apache.nutch.crawl
Partition urls by host, domain name or IP depending on the value of the parameter 'partition.url.mode' which can be 'byHost', 'byDomain' or 'byIP'
URLPartitioner() - Constructor for class org.apache.nutch.crawl.URLPartitioner
 
URLUtil - Class in org.apache.nutch.util
Utility class for URL analysis
URLUtil() - Constructor for class org.apache.nutch.util.URLUtil
 
useHttp11 - Variable in class org.apache.nutch.protocol.http.api.HttpBase
Do we use HTTP/1.1?
useProxy - Variable in class org.apache.nutch.protocol.http.api.HttpBase
Indicates if a proxy is used
useProxy() - Method in class org.apache.nutch.protocol.http.api.HttpBase
 
userAgent - Variable in class org.apache.nutch.protocol.http.api.HttpBase
The Nutch 'User-Agent' request header
UUID_KEY - Static variable in class org.apache.nutch.util.NutchConfiguration
 

V

valueOf(String) - Static method in enum org.apache.nutch.util.domain.DomainStatistics.MyCounter
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum org.apache.nutch.util.domain.DomainSuffix.Status
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum org.apache.nutch.util.domain.TopLevelDomain.Type
Returns the enum constant of this type with the specified name.
values() - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
values() - Static method in enum org.apache.nutch.util.domain.DomainStatistics.MyCounter
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum org.apache.nutch.util.domain.DomainSuffix.Status
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum org.apache.nutch.util.domain.TopLevelDomain.Type
Returns an array containing the constants of this enum type, in the order they are declared.
VERSION - Static variable in class org.apache.nutch.indexer.NutchDocument
 

W

walk(Node, URL, Metadata, Configuration) - Static method in class org.creativecommons.nutch.CCParseFilter.Walker
Scan the document adding attributes to metadata.
WebGraph - Class in org.apache.nutch.scoring.webgraph
Creates three databases, one for inlinks, one for outlinks, and a node database that holds the number of in and outlinks to a url and the current score for the url.
WebGraph() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph
 
WebGraph.OutlinkDb - Class in org.apache.nutch.scoring.webgraph
The OutlinkDb creates a database of all outlinks.
WebGraph.OutlinkDb() - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
Default constructor.
WebGraph.OutlinkDb(Configuration) - Constructor for class org.apache.nutch.scoring.webgraph.WebGraph.OutlinkDb
Configurable constructor.
WORD_COUNT - Static variable in interface org.apache.nutch.metadata.Office
 
WORK_TYPE - Static variable in interface org.apache.nutch.metadata.CreativeCommons
 
WOULDBLOCK - Static variable in class org.apache.nutch.protocol.ProtocolStatus
Request was refused by protocol plugins, because it would block.
WRITABLE_GENERATE_TIME_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
WRITABLE_PROTO_STATUS_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
WRITABLE_REPR_URL_KEY - Static variable in interface org.apache.nutch.metadata.Nutch
 
write(DataOutput) - Method in class org.apache.nutch.crawl.CrawlDatum
 
write(Text, CrawlDatum) - Method in class org.apache.nutch.crawl.CrawlDbReader.CrawlDatumCsvOutputFormat.LineRecordWriter
 
write(DataOutput) - Method in class org.apache.nutch.crawl.Generator.SelectorEntry
 
write(DataOutput) - Method in class org.apache.nutch.crawl.Inlink
 
write(DataOutput) - Method in class org.apache.nutch.crawl.Inlinks
 
write(DataOutput) - Method in class org.apache.nutch.crawl.MapWritable
Deprecated.  
write(DataOutput) - Method in class org.apache.nutch.fetcher.FetcherOutput
 
write(DataOutput) - Method in class org.apache.nutch.indexer.NutchDocument
 
write(DataOutput) - Method in class org.apache.nutch.indexer.NutchField
 
write(NutchDocument) - Method in interface org.apache.nutch.indexer.NutchIndexWriter
 
write(DataOutput) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrInputSplit
 
write(DataOutput) - Method in class org.apache.nutch.indexer.solr.SolrDeleteDuplicates.SolrRecord
 
write(NutchDocument) - Method in class org.apache.nutch.indexer.solr.SolrWriter
 
write(DataOutput) - Method in class org.apache.nutch.metadata.Metadata
 
write(DataOutput) - Method in class org.apache.nutch.metadata.MetaWrapper
 
write(DataOutput) - Method in class org.apache.nutch.parse.Outlink
 
write(DataOutput) - Method in class org.apache.nutch.parse.ParseData
 
write(DataOutput) - Method in class org.apache.nutch.parse.ParseImpl
 
write(DataOutput) - Method in class org.apache.nutch.parse.ParseStatus
 
write(DataOutput) - Method in class org.apache.nutch.parse.ParseText
 
write(DataOutput) - Method in class org.apache.nutch.protocol.Content
 
write(DataOutput) - Method in class org.apache.nutch.protocol.ProtocolStatus
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDatum
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNode
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.LinkDumper.LinkNodes
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.Loops.LoopSet
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.Loops.Route
 
write(DataOutput) - Method in class org.apache.nutch.scoring.webgraph.Node
 
WWW_AUTHENTICATE - Static variable in class org.apache.nutch.protocol.httpclient.HttpAuthenticationFactory
The HTTP Authentication (WWW-Authenticate) header which is returned by a webserver requiring authentication.

X

X_POINT_ID - Static variable in interface org.apache.nutch.indexer.IndexingFilter
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.net.URLFilter
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.net.URLNormalizer
 
X_POINT_ID - Static variable in interface org.apache.nutch.parse.HtmlParseFilter
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.parse.Parser
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.protocol.Protocol
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.scoring.ScoringFilter
The name of the extension point.
X_POINT_ID - Static variable in interface org.apache.nutch.segment.SegmentMergeFilter
The name of the extension point.

Z

zip(byte[]) - Static method in class org.apache.nutch.util.GZIPUtils
Returns an gzipped copy of the input array.
ZipParser - Class in org.apache.nutch.parse.zip
ZipParser class based on MSPowerPointParser class by Stephan Strittmatter.
ZipParser() - Constructor for class org.apache.nutch.parse.zip.ZipParser
Creates a new instance of ZipParser
ZipTextExtractor - Class in org.apache.nutch.parse.zip
 
ZipTextExtractor(Configuration) - Constructor for class org.apache.nutch.parse.zip.ZipTextExtractor
Creates a new instance of ZipTextExtractor

_

__openPassiveDataConnection(int, String) - Method in class org.apache.nutch.protocol.ftp.Client
 
_compare(Object, Object) - Static method in class org.apache.nutch.crawl.SignatureComparator
 
_compare(byte[], int, int, byte[], int, int) - Static method in class org.apache.nutch.crawl.SignatureComparator
 

A B C D E F G H I J K L M N O P R S T U V W X Z _

Copyright © 2011 The Apache Software Foundation