![]() | By default, closes the input Reader. |
![]() | Determines whether the specified Object is equal to the current Object. |
![]() | Serves as a hash function for a particular type, suitable for use in hashing algorithms and data structures like a hash table. |
![]() | Gets the Type of the current instance. |
![]() | Returns the next token in the stream, or null at EOS. |
![]() | Returns a String that represents the current Object. |
![]() | Allows an Object to attempt to free resources and perform other cleanup operations before the Object is reclaimed by garbage collection. |
![]() | Creates a shallow copy of the current Object. |
![]() | The text source for this Tokenizer. |
![]() | Returns true iff a character should be included in a token. This tokenizer generates as tokens adjacent sequences of characters which satisfy this predicate. Characters for which this is false are used to define token boundaries and are not included in tokens. |
![]() | Called on each token character to normalize it before it is added to the token. The default implementation does nothing. Subclasses may use this to, e.g., lowercase tokens. |