/[Apache-SVN]
ViewVC logotype

Revision 1548762

Jump to revision: Previous Next
Author: sarowe
Date: Sat Dec 7 00:05:16 2013 UTC (10 years, 4 months ago)
Changed paths: 46
Log Message: 
LUCENE-5357: Upgrade StandardTokenizer and UAX29URLEmailTokenizer to Unicode 6.3; update UAX29URLEmailTokenizer's recognized top level domains in URLs and Emails from the IANA Root Zone Database; add std40/StandardTokenizerImpl40 and std40/UAX29URLEmailTokenizerImpl40, for backcompat from 4.0->4.6.  (merged trunk r1548595 and r1548746)

Changed paths

Path Details
Directorylucene/dev/branches/branch_4x/ modified , props changed
Directorylucene/dev/branches/branch_4x/lucene/ modified , props changed
Directorylucene/dev/branches/branch_4x/lucene/CHANGES.txt modified , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/ modified , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/build.xml modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/charfilter/HTMLStripCharFilter.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/charfilter/HTMLStripCharFilter.jflex modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ClassicTokenizerImpl.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ClassicTokenizerImpl.jflex modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/READ_BEFORE_REGENERATING.txt modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/SUPPLEMENTARY.jflex-macro modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/StandardTokenizer.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizer.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizerImpl.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizerImpl.jflex modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std31/StandardTokenizerImpl31.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std31/UAX29URLEmailTokenizerImpl31.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std34/StandardTokenizerImpl34.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std34/UAX29URLEmailTokenizerImpl34.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std36/UAX29URLEmailTokenizerImpl36.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/ added
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/ASCIITLD.jflex-macro
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/ASCIITLD.jflex-macro, r1548600) 		added , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/SUPPLEMENTARY.jflex-macro
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/SUPPLEMENTARY.jflex-macro, r1547501) 		added , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/StandardTokenizerImpl40.java
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.java, r1548600) 		added , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/StandardTokenizerImpl40.jflex
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/StandardTokenizerImpl.jflex, r1548600) 		added , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/UAX29URLEmailTokenizerImpl40.java
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizerImpl.java, r1548600) 		added , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/UAX29URLEmailTokenizerImpl40.jflex
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizerImpl.jflex, r1548600) 		added , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/std40/package.html
(Copied from lucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/standard/package.html, r1548600) 		added , text changed , props changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/wikipedia/WikipediaTokenizerImpl.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/java/org/apache/lucene/analysis/wikipedia/WikipediaTokenizerImpl.jflex modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestStandardAnalyzer.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/WordBreakTestUnicode_6_1_0.java deleted
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/WordBreakTestUnicode_6_3_0.java
(Copied from lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/WordBreakTestUnicode_6_3_0.java, r1548595) 		added
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/email.addresses.from.random.text.with.email.addresses.txt modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/generateJavaUnicodeWordBreakTest.pl modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/random.text.with.email.addresses.txt modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/random.text.with.urls.txt modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/urls.from.random.text.with.urls.txt modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/test/org/apache/lucene/analysis/standard/TestUAX29URLEmailTokenizerFactory.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/common/src/tools/java/org/apache/lucene/analysis/standard/GenerateJflexTLDMacros.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/analysis/icu/src/tools/java/org/apache/lucene/analysis/icu/GenerateJFlexSupplementaryMacros.java modified , text changed
Directorylucene/dev/branches/branch_4x/lucene/common-build.xml modified , text changed , props changed
infrastructure at apache.org
 ViewVC Help
Powered by ViewVC 1.1.26