Lucene.Net
3.0.3
Lucene.Net is a .NET port of the Java Lucene Indexing Library
|
Removes stop words from a token stream. More...
Inherits Lucene.Net.Analysis.TokenFilter.
Public Member Functions | |
StopFilter (bool enablePositionIncrements, TokenStream input, ISet< string > stopWords, bool ignoreCase) | |
Construct a token stream filtering the given input. If stopWords is an instance of CharArraySet (true if makeStopSet() was used to construct the set) it will be directly used and ignoreCase will be ignored since CharArraySet directly controls case sensitivity. If stopWords is not an instance of CharArraySet, a new CharArraySet will be constructed and ignoreCase will be used to specify the case sensitivity of that set. | |
StopFilter (bool enablePositionIncrements, TokenStream @in, ISet< string > stopWords) | |
Constructs a filter which removes words from the input TokenStream that are named in the Set. | |
override bool | IncrementToken () |
Returns the next input Token whose term() is not a stop word. | |
Public Member Functions inherited from Lucene.Net.Analysis.TokenFilter | |
override void | End () |
Performs end-of-stream operations, if any, and calls then end() on the input TokenStream.NOTE: Be sure to call super.end() first when overriding this method. | |
override void | Reset () |
Reset the filter as well as the input TokenStream. | |
Public Member Functions inherited from Lucene.Net.Analysis.TokenStream | |
void | Close () |
Releases resources associated with this stream. | |
void | Dispose () |
Public Member Functions inherited from Lucene.Net.Util.AttributeSource | |
AttributeSource () | |
An AttributeSource using the default attribute factory AttributeSource.AttributeFactory.DEFAULT_ATTRIBUTE_FACTORY. | |
AttributeSource (AttributeSource input) | |
An AttributeSource that uses the same attributes as the supplied one. | |
AttributeSource (AttributeFactory factory) | |
An AttributeSource using the supplied AttributeFactory for creating new IAttribute instances. | |
virtual IEnumerable< Type > | GetAttributeTypesIterator () |
Returns a new iterator that iterates the attribute classes in the same order they were added in. Signature for Java 1.5: public Iterator<Class<? extends Attribute>> getAttributeClassesIterator() | |
virtual IEnumerable< Attribute > | GetAttributeImplsIterator () |
Returns a new iterator that iterates all unique Attribute implementations. This iterator may contain less entries that GetAttributeTypesIterator, if one instance implements more than one Attribute interface. Signature for Java 1.5: public Iterator<AttributeImpl> getAttributeImplsIterator() | |
virtual void | AddAttributeImpl (Attribute att) |
Expert: Adds a custom AttributeImpl instance with one or more Attribute interfaces. | |
virtual T | AddAttribute< T > () |
The caller must pass in a Class<? extends Attribute> value. This method first checks if an instance of that class is already in this AttributeSource and returns it. Otherwise a new instance is created, added to this AttributeSource and returned. | |
virtual bool | HasAttribute< T > () |
The caller must pass in a Class<? extends Attribute> value. Returns true, iff this AttributeSource contains the passed-in Attribute. | |
virtual T | GetAttribute< T > () |
The caller must pass in a Class<? extends Attribute> value. Returns the instance of the passed in Attribute contained in this AttributeSource | |
virtual void | ClearAttributes () |
Resets all Attributes in this AttributeSource by calling Attribute.Clear() on each Attribute implementation. | |
virtual State | CaptureState () |
Captures the state of all Attributes. The return value can be passed to RestoreState to restore the state of this or another AttributeSource. | |
virtual void | RestoreState (State state) |
Restores this state by copying the values of all attribute implementations that this state contains into the attributes implementations of the targetStream. The targetStream must contain a corresponding instance for each argument contained in this state (e.g. it is not possible to restore the state of an AttributeSource containing a TermAttribute into a AttributeSource using a Token instance as implementation). | |
override int | GetHashCode () |
override bool | Equals (System.Object obj) |
override System.String | ToString () |
virtual AttributeSource | CloneAttributes () |
Performs a clone of all Attribute instances returned in a new AttributeSource instance. This method can be used to e.g. create another TokenStream with exactly the same attributes (using AttributeSource(AttributeSource)) | |
Static Public Member Functions | |
static ISet< string > | MakeStopSet (params string[] stopWords) |
Builds a Set from an array of stop words, appropriate for passing into the StopFilter constructor. This permits this stopWords construction to be cached once when an Analyzer is constructed. | |
static ISet< string > | MakeStopSet (IList< object > stopWords) |
Builds a Set from an array of stop words, appropriate for passing into the StopFilter constructor. This permits this stopWords construction to be cached once when an Analyzer is constructed. | |
static ISet< string > | MakeStopSet (string[] stopWords, bool ignoreCase) |
| |
static ISet< string > | MakeStopSet (IList< object > stopWords, bool ignoreCase) |
| |
static bool | GetEnablePositionIncrementsVersionDefault (Version matchVersion) |
Returns version-dependent default for enablePositionIncrements. Analyzers that embed StopFilter use this method when creating the StopFilter. Prior to 2.9, this returns false. On 2.9 or later, it returns true. | |
Properties | |
bool | EnablePositionIncrements [get, set] |
If true , this StopFilter will preserve positions of the incoming tokens (ie, accumulate and set position increments of the removed stop tokens). Generally, true is best as it does not lose information (positions of the original tokens) during indexing. | |
Additional Inherited Members | |
Protected Member Functions inherited from Lucene.Net.Analysis.TokenFilter | |
override void | Dispose (bool disposing) |
Removes stop words from a token stream.
Definition at line 30 of file StopFilter.cs.
Lucene.Net.Analysis.StopFilter.StopFilter | ( | bool | enablePositionIncrements, |
TokenStream | input, | ||
ISet< string > | stopWords, | ||
bool | ignoreCase | ||
) |
Construct a token stream filtering the given input. If stopWords
is an instance of CharArraySet (true if makeStopSet()
was used to construct the set) it will be directly used and ignoreCase
will be ignored since CharArraySet
directly controls case sensitivity. If stopWords
is not an instance of CharArraySet, a new CharArraySet will be constructed and ignoreCase
will be used to specify the case sensitivity of that set.
enablePositionIncrements | true if token positions should record the removed stop words |
input | Input TokenStream |
stopWords | A Set of strings or strings or char[] or any other ToString()-able set representing the stopwords |
ignoreCase | if true, all words are lower cased first |
Definition at line 52 of file StopFilter.cs.
Lucene.Net.Analysis.StopFilter.StopFilter | ( | bool | enablePositionIncrements, |
TokenStream @ | in, | ||
ISet< string > | stopWords | ||
) |
Constructs a filter which removes words from the input TokenStream that are named in the Set.
enablePositionIncrements | true if token positions should record the removed stop words |
in | Input stream |
stopWords | A Set of strings or char[] or any other ToString()-able set representing the stopwords |
Definition at line 76 of file StopFilter.cs.
|
static |
Returns version-dependent default for enablePositionIncrements. Analyzers that embed StopFilter use this method when creating the StopFilter. Prior to 2.9, this returns false. On 2.9 or later, it returns true.
Definition at line 152 of file StopFilter.cs.
|
virtual |
Returns the next input Token whose term() is not a stop word.
Implements Lucene.Net.Analysis.TokenStream.
Definition at line 128 of file StopFilter.cs.
|
static |
Builds a Set from an array of stop words, appropriate for passing into the StopFilter constructor. This permits this stopWords construction to be cached once when an Analyzer is constructed.
passing false to ignoreCase
Definition at line 87 of file StopFilter.cs.
|
static |
Builds a Set from an array of stop words, appropriate for passing into the StopFilter constructor. This permits this stopWords construction to be cached once when an Analyzer is constructed.
stopWords | A list of strings or char[] or any other ToString()-able list representing the stop words |
passing false to ignoreCase
Definition at line 99 of file StopFilter.cs.
|
static |
stopWords | An array of stopwords |
ignoreCase | If true, all words are lower cased first. |
Definition at line 108 of file StopFilter.cs.
|
static |
stopWords | A List of Strings or char[] or any other toString()-able list representing the stopwords |
ignoreCase | if true, all words are lower cased first |
Definition at line 119 of file StopFilter.cs.
|
getset |
If true
, this StopFilter will preserve positions of the incoming tokens (ie, accumulate and set position increments of the removed stop tokens). Generally, true
is best as it does not lose information (positions of the original tokens) during indexing.
When set, when a token is stopped (omitted), the position increment of the following token is incremented.
NOTE: be sure to also set QueryParser.EnablePositionIncrements if you use QueryParser to create queries.
Definition at line 173 of file StopFilter.cs.