public final class DruidStorageHandlerUtils extends Object
Modifier and Type | Class and Description |
---|---|
static interface |
DruidStorageHandlerUtils.DataPusher
Simple interface for retry operations
|
Modifier and Type | Field and Description |
---|---|
static com.google.common.collect.Interner<io.druid.timeline.DataSegment> |
DATA_SEGMENT_INTERNER
Generic Interner implementation used to read segments object from metadata storage
|
static io.druid.segment.IndexIO |
INDEX_IO
Used by druid to perform IO on indexes
|
static io.druid.segment.IndexMergerV9 |
INDEX_MERGER_V9
Used by druid to merge indexes
|
static com.fasterxml.jackson.databind.ObjectMapper |
JSON_MAPPER
Mapper to use to serialize/deserialize Druid objects (JSON)
|
static com.fasterxml.jackson.databind.ObjectMapper |
SMILE_MAPPER
Mapper to use to serialize/deserialize Druid objects (SMILE)
|
Constructor and Description |
---|
DruidStorageHandlerUtils() |
Modifier and Type | Method and Description |
---|---|
static void |
addDependencyJars(org.apache.hadoop.conf.Configuration conf,
Class<?>... classes) |
static com.metamx.http.client.Request |
createRequest(String address,
io.druid.query.BaseQuery<?> query)
Method that creates a request for Druid JSON query (using SMILE).
|
static boolean |
disableDataSource(io.druid.metadata.SQLMetadataConnector connector,
io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig,
String dataSource) |
static Collection<String> |
getAllDataSourceNames(io.druid.metadata.SQLMetadataConnector connector,
io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig) |
static List<io.druid.timeline.DataSegment> |
getDataSegmentList(io.druid.metadata.SQLMetadataConnector connector,
io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig,
String dataSource) |
static List<io.druid.timeline.DataSegment> |
getPublishedSegments(org.apache.hadoop.fs.Path taskDir,
org.apache.hadoop.conf.Configuration conf) |
static String |
getURL(com.metamx.http.client.HttpClient client,
URL url) |
static org.apache.hadoop.fs.Path |
makeSegmentDescriptorOutputPath(io.druid.timeline.DataSegment pushedSegment,
org.apache.hadoop.fs.Path segmentsDescriptorDir) |
static InputStream |
submitRequest(com.metamx.http.client.HttpClient client,
com.metamx.http.client.Request request)
Method that submits a request to an Http address and retrieves the result.
|
static void |
writeSegmentDescriptor(org.apache.hadoop.fs.FileSystem outputFS,
io.druid.timeline.DataSegment segment,
org.apache.hadoop.fs.Path descriptorPath)
This function will write to filesystem serialized from of segment descriptor
if an existing file exists it will try to replace it.
|
public static final com.fasterxml.jackson.databind.ObjectMapper JSON_MAPPER
public static final com.fasterxml.jackson.databind.ObjectMapper SMILE_MAPPER
public static final io.druid.segment.IndexIO INDEX_IO
public static final io.druid.segment.IndexMergerV9 INDEX_MERGER_V9
public static final com.google.common.collect.Interner<io.druid.timeline.DataSegment> DATA_SEGMENT_INTERNER
public static com.metamx.http.client.Request createRequest(String address, io.druid.query.BaseQuery<?> query) throws IOException
address
- query
- IOException
public static InputStream submitRequest(com.metamx.http.client.HttpClient client, com.metamx.http.client.Request request) throws IOException
client
- request
- IOException
public static String getURL(com.metamx.http.client.HttpClient client, URL url) throws IOException
IOException
public static List<io.druid.timeline.DataSegment> getPublishedSegments(org.apache.hadoop.fs.Path taskDir, org.apache.hadoop.conf.Configuration conf) throws IOException
taskDir
- path to the directory containing the segments descriptor info
the descriptor path will be .../workingPath/task_id/DruidStorageHandler.SEGMENTS_DESCRIPTOR_DIR_NAME
/*.jsonconf
- hadoop conf to get the file systemIOException
- can be for the case we did not produce data.public static void writeSegmentDescriptor(org.apache.hadoop.fs.FileSystem outputFS, io.druid.timeline.DataSegment segment, org.apache.hadoop.fs.Path descriptorPath) throws IOException
outputFS
- filesystemsegment
- DataSegment objectdescriptorPath
- pathIOException
public static Collection<String> getAllDataSourceNames(io.druid.metadata.SQLMetadataConnector connector, io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig)
connector
- SQL metadata connector to the metadata storagemetadataStorageTablesConfig
- Table configpublic static boolean disableDataSource(io.druid.metadata.SQLMetadataConnector connector, io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig, String dataSource)
connector
- SQL connector to metadatametadataStorageTablesConfig
- Tables configurationdataSource
- Name of data sourcepublic static List<io.druid.timeline.DataSegment> getDataSegmentList(io.druid.metadata.SQLMetadataConnector connector, io.druid.metadata.MetadataStorageTablesConfig metadataStorageTablesConfig, String dataSource)
connector
- SQL connector to metadatametadataStorageTablesConfig
- Tables configurationdataSource
- Name of data sourcepublic static org.apache.hadoop.fs.Path makeSegmentDescriptorOutputPath(io.druid.timeline.DataSegment pushedSegment, org.apache.hadoop.fs.Path segmentsDescriptorDir)
pushedSegment
- segmentsDescriptorDir
- public static void addDependencyJars(org.apache.hadoop.conf.Configuration conf, Class<?>... classes) throws IOException
IOException
Copyright © 2021 The Apache Software Foundation. All rights reserved.