- All Implemented Interfaces:
- org.carrot2.text.linguistic.ITokenizerFactory
public class LuceneCarrot2TokenizerFactory
extends Object
implements org.carrot2.text.linguistic.ITokenizerFactory
An implementation of Carrot2's ITokenizerFactory
based on Lucene's
Smart Chinese tokenizer. If Smart Chinese tokenizer is not available in
classpath at runtime, the default Carrot2's tokenizer is used. Should the
Lucene APIs need to change, the changes can be made in this class.
- WARNING: This API is experimental and might change in incompatible ways in the next release.