LetterTokenizer Constructor | Construct a new LetterTokenizer. |
Close (inherited from Tokenizer) | By default, closes the input Reader. |
Equals (inherited from Object) | Determines whether the specified Object is equal to the current Object. |
GetHashCode (inherited from Object) | Serves as a hash function for a particular type, suitable for use in hashing algorithms and data structures like a hash table. |
GetType (inherited from Object) | Gets the Type of the current instance. |
Next (inherited from CharTokenizer) | Returns the next token in the stream, or null at EOS. |
ToString (inherited from Object) | Returns a String that represents the current Object. |
Finalize (inherited from Object) | Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection. |
MemberwiseClone (inherited from Object) | Creates a shallow copy of the current Object. |
input (inherited from Tokenizer) | The text source for this Tokenizer. |
IsTokenChar | Collects only characters which satisfy {@link Character#isLetter(char)}. |
Normalize (inherited from CharTokenizer) | Called on each token character to normalize it before it is added to the token. The default implementation does nothing. Subclasses may use this to, e.g., lowercase tokens. |