/[Apache-SVN]
ViewVC logotype

Revision 1662436

Jump to revision: Previous Next
Author: shalin
Date: Thu Feb 26 12:43:00 2015 UTC (9 years, 1 month ago)
Changed paths: 9
Log Message: 
SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor:
- Added langid.maxFieldValueChars and langid.maxTotalChars params to limit
  input, by default 10k and 20k chars, respectively.
- Moved input concatenation to Tika implementation; the langdetect
  implementation instead appends each input piece via the langdetect API.

Changed paths

Path Details
Directorylucene/dev/branches/lucene_solr_4_10/ modified , props changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/ modified , props changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/CHANGES.txt modified , text changed , props changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/ modified , props changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessor.java modified , text changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangIdParams.java modified , text changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/langid/src/java/org/apache/solr/update/processor/LanguageIdentifierUpdateProcessor.java modified , text changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessor.java modified , text changed
Directorylucene/dev/branches/lucene_solr_4_10/solr/contrib/langid/src/test/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactoryTest.java modified , text changed
infrastructure at apache.org
 ViewVC Help
Powered by ViewVC 1.1.26