/// Typical implementations first build a Tokenizer, which breaks the stream of /// characters from the Reader into raw Tokens. One or more TokenFilters may /// then be applied to the output of the Tokenizer. ///
/// WARNING: You must override one of the methods defined by this class in your /// subclass or the Analyzer will enter an infinite loop. ///