Called on each token character to normalize it before it is added to the token. The default implementation does nothing. Subclasses may use this to, e.g., lowercase tokens.
CharTokenizer Class | Lucene.Net.Analysis Namespace