Class Lucene50CompressingStoredFieldsFormat
StoredFieldsFormat
that compresses documents in chunks in order to improve the
compression ratio.
For a chunk size of chunkSize bytes, this StoredFieldsFormat
does not
support documents larger than (231 - chunkSize
) bytes.
For optimal performance, you should use a MergePolicy
that returns segments that have
the biggest byte size first.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Field Summary
Modifier and TypeFieldDescriptionprotected final int
block shiftprotected final int
chunk sizeprotected final CompressionMode
compression modeprotected final String
format nameprotected final int
max docs per chunkprotected final String
segment suffix -
Constructor Summary
ConstructorDescriptionLucene50CompressingStoredFieldsFormat
(String formatName, String segmentSuffix, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a newLucene50CompressingStoredFieldsFormat
.Lucene50CompressingStoredFieldsFormat
(String formatName, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a newLucene50CompressingStoredFieldsFormat
with an empty segment suffix. -
Method Summary
Modifier and TypeMethodDescriptionfieldsReader
(Directory directory, SegmentInfo si, FieldInfos fn, IOContext context) fieldsWriter
(Directory directory, SegmentInfo si, IOContext context) toString()
-
Field Details
-
formatName
format name -
segmentSuffix
segment suffix -
compressionMode
compression mode -
chunkSize
protected final int chunkSizechunk size -
maxDocsPerChunk
protected final int maxDocsPerChunkmax docs per chunk -
blockShift
protected final int blockShiftblock shift
-
-
Constructor Details
-
Lucene50CompressingStoredFieldsFormat
public Lucene50CompressingStoredFieldsFormat(String formatName, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a newLucene50CompressingStoredFieldsFormat
with an empty segment suffix.- See Also:
-
Lucene50CompressingStoredFieldsFormat
public Lucene50CompressingStoredFieldsFormat(String formatName, String segmentSuffix, CompressionMode compressionMode, int chunkSize, int maxDocsPerChunk, int blockShift) Create a newLucene50CompressingStoredFieldsFormat
.formatName
is the name of the format. This name will be used in the file formats to performcodec header checks
.segmentSuffix
is the segment suffix. This suffix is added to the result file name only if it's not the empty string.The
compressionMode
parameter allows you to choose between compression algorithms that have various compression and decompression speeds so that you can pick the one that best fits your indexing and searching throughput. You should never instantiate twoLucene50CompressingStoredFieldsFormat
s that have the same name but differentCompressionMode
s.chunkSize
is the minimum byte size of a chunk of documents. A value of1
can make sense if there is redundancy across fields.maxDocsPerChunk
is an upperbound on how many docs may be stored in a single chunk. This is to bound the cpu costs for highly compressible data.Higher values of
chunkSize
should improve the compression ratio but will require more memory at indexing time and might make document loading a little slower (depending on the size of your OS cache compared to the size of your index).- Parameters:
formatName
- the name of theStoredFieldsFormat
compressionMode
- theCompressionMode
to usechunkSize
- the minimum number of bytes of a single chunk of stored documentsmaxDocsPerChunk
- the maximum number of documents in a single chunkblockShift
- the log in base 2 of number of chunks to store in an index block- See Also:
-
-
Method Details
-
fieldsReader
public StoredFieldsReader fieldsReader(Directory directory, SegmentInfo si, FieldInfos fn, IOContext context) throws IOException - Specified by:
fieldsReader
in classStoredFieldsFormat
- Throws:
IOException
-
fieldsWriter
public StoredFieldsWriter fieldsWriter(Directory directory, SegmentInfo si, IOContext context) throws IOException - Specified by:
fieldsWriter
in classStoredFieldsFormat
- Throws:
IOException
-
toString
-