org.apache.nutch.analysis.lang
Class LanguageIndexingFilter
java.lang.Object
org.apache.nutch.analysis.lang.LanguageIndexingFilter
- All Implemented Interfaces:
- Configurable, IndexingFilter, FieldPluggable, Pluggable
public class LanguageIndexingFilter
- extends Object
- implements IndexingFilter
An IndexingFilter
that adds a
lang
(language) field to the document.
It tries to find the language of the document by checking
if HTMLLanguageParser
has added some language
information
- Author:
- Sami Siren, Jerome Charron
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LanguageIndexingFilter
public LanguageIndexingFilter()
- Constructs a new Language Indexing Filter.
filter
public NutchDocument filter(NutchDocument doc,
String url,
WebPage page)
throws IndexingException
- Description copied from interface:
IndexingFilter
- Adds fields or otherwise modifies the document that will be indexed for a
parse. Unwanted documents can be removed from indexing by returning a null value.
- Specified by:
filter
in interface IndexingFilter
- Parameters:
doc
- document instance for collecting fieldsurl
- page url
- Returns:
- modified (or a new) document instance, or null (meaning the document
should be discarded)
- Throws:
IndexingException
getFields
public Collection<WebPage.Field> getFields()
- Specified by:
getFields
in interface FieldPluggable
addIndexBackendOptions
public void addIndexBackendOptions(Configuration conf)
setConf
public void setConf(Configuration conf)
- Specified by:
setConf
in interface Configurable
getConf
public Configuration getConf()
- Specified by:
getConf
in interface Configurable
Copyright © 2012 The Apache Software Foundation