| Log Message: |
Distinguish between positions and length when indexing a field. The
length is now defined as the total number of tokens, not the final
position. Length is used for score normalization
(Similarity.lengthNorm()) and for controlling memory usage
(IndexWriter.maxFieldLength). In both cases the total number of
tokens is more reasonable than the final position. Position is used
in phrase searching (see PhraseQuery and Token.setPositionIncrement()).
|
lucene/java/trunk/src/java/org/apache/lucene/index/DocumentWriter.java