Public Static Fields
Public Instance Constructors
StandardTokenizer
|
Overloaded. Initializes a new instance of the StandardTokenizer class. |
Public Instance Methods
| Close (inherited from Tokenizer) | By default, closes the input Reader. |
| Equals (inherited from Object) | Determines whether the specified Object is equal to the current Object. |
| GetHashCode (inherited from Object) | Serves as a hash function for a particular type. GetHashCode is suitable for use in hashing algorithms and data structures like a hash table. |
| GetMaxTokenLength | |
| GetType (inherited from Object) | Gets the Type of the current instance. |
| IsReplaceInvalidAcronym | Prior to https://issues.apache.org/jira/browse/LUCENE-1068, StandardTokenizer mischaracterized as acronyms tokens like www.abc.com when they should have been labeled as hosts instead. |
| Next | Overloaded. |
| Next (inherited from TokenStream) | Overloaded. Returns the next token in the stream, or null at EOS. @deprecated The returned Token is a "full private copy" (not re-used across calls to next()) but will be slower than calling {@link #Next(Token)} instead.. |
| Reset | Overloaded. |
| SetMaxTokenLength | Set the max allowed token length. Any token longer than this is skipped. |
| SetReplaceInvalidAcronym | |
| ToString (inherited from Object) | Returns a String that represents the current Object. |
Protected Instance Methods
 Finalize (inherited from Object) | Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection. |
| MemberwiseClone (inherited from Object) | Creates a shallow copy of the current Object. |
Protected Internal Instance Fields
 input (inherited from Tokenizer) | The text source for this Tokenizer. |
See Also
StandardTokenizer Class | Lucene.Net.Analysis.Standard Namespace