/[Apache-SVN]

Revision 1714354

Jump to revision:
Author:	uschindler
Date:	Sat Nov 14 19:21:20 2015 UTC (8 years, 5 months ago)
Changed paths:	14
Log Message:	LUCENE-6874: Add a new UnicodeWhitespaceTokenizer to analysis/common that uses Unicode character properties extracted from ICU4J to tokenize text on whitespace

Changed paths

Path	Details
lucene/dev/trunk/lucene/CHANGES.txt	modified , text changed
lucene/dev/trunk/lucene/analysis/common/build.xml	modified , text changed
lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/core/UnicodeWhitespaceAnalyzer.java	added
lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/core/UnicodeWhitespaceTokenizer.java	added
lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/core/WhitespaceTokenizer.java	modified , text changed
lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/core/WhitespaceTokenizerFactory.java	modified , text changed
lucene/dev/trunk/lucene/analysis/common/src/java/org/apache/lucene/analysis/util/UnicodeProps.java	added
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestAllAnalyzersHaveFactories.java	modified , text changed
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestAnalyzers.java	modified , text changed
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUnicodeWhitespaceTokenizer.java	added
lucene/dev/trunk/lucene/analysis/common/src/tools/groovy/	added
lucene/dev/trunk/lucene/analysis/common/src/tools/groovy/generate-unicode-data.groovy	added
lucene/dev/trunk/lucene/benchmark/conf/wstok.alg	added
lucene/dev/trunk/lucene/benchmark/src/java/org/apache/lucene/benchmark/utils/ExtractReuters.java	modified , text changed

infrastructure at apache.org	ViewVC Help
Powered by ViewVC 1.1.26