public class ExportWriter extends Object implements SolrCore.RawWriter, Closeable
ExportWriter
gathers and sorts the documents for a core using "stream sorting".
Stream sorting works by repeatedly processing and modifying a bitmap of matching documents. Each pass over the
bitmap identifies the smallest docs (default is DEFAULT_BATCH_SIZE
) that haven't been sent yet and stores them in a
Priority Queue. They are then exported (written across the wire) and marked as sent (unset in the bitmap).
This process repeats until all matching documents have been sent.
This streaming approach is light on memory (only up to 2x batch size documents are ever stored in memory at
once), and it allows ExportWriter
to scale well with regard to numDocs.
Modifier and Type | Class and Description |
---|---|
static class |
ExportWriter.IgnoreException |
Modifier and Type | Field and Description |
---|---|
static String |
BATCH_SIZE_PARAM |
static int |
DEFAULT_BATCH_SIZE |
Constructor and Description |
---|
ExportWriter(SolrQueryRequest req,
SolrQueryResponse res,
String wt,
StreamContext initialStreamContext,
SolrMetricsContext solrMetricsContext,
String metricsPath,
ExportHandler exportHandler) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
String |
getContentType() |
org.apache.solr.handler.export.FieldWriter[] |
getFieldWriters(String[] fields,
SolrIndexSearcher searcher) |
void |
write(OutputStream os) |
protected void |
writeDocs(SolrQueryRequest req,
OutputStream os,
IteratorWriter.ItemWriter writer,
Sort sort) |
protected void |
writeException(Exception e,
PushWriter w,
boolean logException) |
public static final String BATCH_SIZE_PARAM
public static final int DEFAULT_BATCH_SIZE
public ExportWriter(SolrQueryRequest req, SolrQueryResponse res, String wt, StreamContext initialStreamContext, SolrMetricsContext solrMetricsContext, String metricsPath, ExportHandler exportHandler)
public String getContentType()
getContentType
in interface SolrCore.RawWriter
public void close() throws IOException
close
in interface Closeable
close
in interface AutoCloseable
IOException
protected void writeException(Exception e, PushWriter w, boolean logException) throws IOException
IOException
public void write(OutputStream os) throws IOException
write
in interface SolrCore.RawWriter
IOException
protected void writeDocs(SolrQueryRequest req, OutputStream os, IteratorWriter.ItemWriter writer, Sort sort) throws IOException
IOException
public org.apache.solr.handler.export.FieldWriter[] getFieldWriters(String[] fields, SolrIndexSearcher searcher) throws IOException
IOException
Copyright © 2000-2021 Apache Software Foundation. All Rights Reserved.