The methods of the Tokenizer class are listed below. For a complete list of Tokenizer class members, see the Tokenizer Members topic.
Public Instance Methods
 Close | By default, closes the input Reader. |
| Equals (inherited from Object) | Determines whether the specified Object is equal to the current Object. |
| GetHashCode (inherited from Object) | Serves as a hash function for a particular type. GetHashCode is suitable for use in hashing algorithms and data structures like a hash table. |
| GetType (inherited from Object) | Gets the Type of the current instance. |
| Next (inherited from TokenStream) | Overloaded. Returns the next token in the stream, or null at EOS. @deprecated The returned Token is a "full private copy" (not re-used across calls to next()) but will be slower than calling {@link #Next(Token)} instead.. |
| Reset | Overloaded. Expert: Reset the tokenizer to a new reader. Typically, an analyzer (in its reusableTokenStream method) will use this to re-use a previously created tokenizer. |
| Reset (inherited from TokenStream) | Overloaded. Resets this stream to the beginning. This is an optional operation, so subclasses may or may not implement this method. Reset() is not needed for the standard indexing process. However, if the Tokens of a TokenStream are intended to be consumed more than once, it is necessary to implement reset(). once, it is necessary to implement reset(). Note that if your TokenStream caches tokens and feeds them back again after a reset, it is imperative that you clone the tokens when you store them away (on the first pass) as well as when you return them (on future passes after reset()). |
| ToString (inherited from Object) | Returns a String that represents the current Object. |
Protected Instance Methods
 Finalize (inherited from Object) | Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection. |
| MemberwiseClone (inherited from Object) | Creates a shallow copy of the current Object. |
See Also
Tokenizer Class | Lucene.Net.Analysis Namespace