public class AvroSerdeUtils extends Object
Modifier and Type | Class and Description |
---|---|
static class |
AvroSerdeUtils.AvroTableProperties
Enum container for all avro table properties.
|
Modifier and Type | Field and Description |
---|---|
static String |
AVRO_SERDE_SCHEMA
Deprecated.
|
static String |
EXCEPTION_MESSAGE |
static String |
SCHEMA_DOC
Deprecated.
|
static String |
SCHEMA_LITERAL
Deprecated.
|
static String |
SCHEMA_NAME
Deprecated.
|
static String |
SCHEMA_NAMESPACE
Deprecated.
|
static String |
SCHEMA_NONE |
static String |
SCHEMA_RETRIEVER
Deprecated.
|
static String |
SCHEMA_URL
Deprecated.
|
Constructor and Description |
---|
AvroSerdeUtils() |
Modifier and Type | Method and Description |
---|---|
static org.apache.avro.Schema |
determineSchemaOrThrowException(org.apache.hadoop.conf.Configuration conf,
Properties properties)
Determine the schema to that's been provided for Avro serde work.
|
static Buffer |
getBufferFromBytes(byte[] input) |
static Buffer |
getBufferFromDecimal(org.apache.hadoop.hive.common.type.HiveDecimal dec,
int scale) |
static byte[] |
getBytesFromByteBuffer(ByteBuffer byteBuffer) |
static org.apache.hadoop.hive.common.type.HiveDecimal |
getHiveDecimalFromByteBuffer(ByteBuffer byteBuffer,
int scale) |
static int |
getIntFromSchema(org.apache.avro.Schema schema,
String name) |
static org.apache.avro.Schema |
getOtherTypeFromNullableType(org.apache.avro.Schema unionSchema)
If the union schema is a nullable union, get the schema for the non-nullable type.
|
static org.apache.avro.Schema |
getSchemaFor(File file) |
static org.apache.avro.Schema |
getSchemaFor(InputStream stream) |
static org.apache.avro.Schema |
getSchemaFor(String str) |
static org.apache.avro.Schema |
getSchemaFor(URL url) |
protected static org.apache.avro.Schema |
getSchemaFromFS(String schemaFSUrl,
org.apache.hadoop.conf.Configuration conf) |
static void |
handleAlterTableForAvro(HiveConf conf,
String serializationLib,
Map<String,String> parameters)
Called on specific alter table events, removes schema url and schema literal from given tblproperties
After the change, HMS solely will be responsible for handling the schema
|
static boolean |
insideMRJob(org.apache.hadoop.mapred.JobConf job)
Determine if we're being executed from within an MR job or as part
of a select * statement.
|
static boolean |
isNullableType(org.apache.avro.Schema schema)
Determine if an Avro schema is of type Union[T, NULL].
|
@Deprecated public static final String SCHEMA_LITERAL
@Deprecated public static final String SCHEMA_URL
@Deprecated public static final String SCHEMA_NAMESPACE
@Deprecated public static final String SCHEMA_NAME
@Deprecated public static final String SCHEMA_DOC
@Deprecated public static final String AVRO_SERDE_SCHEMA
@Deprecated public static final String SCHEMA_RETRIEVER
public static final String SCHEMA_NONE
public static final String EXCEPTION_MESSAGE
public static org.apache.avro.Schema determineSchemaOrThrowException(org.apache.hadoop.conf.Configuration conf, Properties properties) throws IOException, AvroSerdeException
properties
- containing a key pointing to the schema, one way or anotherIOException
- if error while trying to read the schema from another locationAvroSerdeException
- if unable to find a schema or pointer to it in the propertiesprotected static org.apache.avro.Schema getSchemaFromFS(String schemaFSUrl, org.apache.hadoop.conf.Configuration conf) throws IOException, URISyntaxException
IOException
URISyntaxException
public static boolean isNullableType(org.apache.avro.Schema schema)
public static org.apache.avro.Schema getOtherTypeFromNullableType(org.apache.avro.Schema unionSchema)
public static boolean insideMRJob(org.apache.hadoop.mapred.JobConf job)
job
- that contains things that are or are not set in a jobpublic static Buffer getBufferFromBytes(byte[] input)
public static Buffer getBufferFromDecimal(org.apache.hadoop.hive.common.type.HiveDecimal dec, int scale)
public static byte[] getBytesFromByteBuffer(ByteBuffer byteBuffer)
public static org.apache.hadoop.hive.common.type.HiveDecimal getHiveDecimalFromByteBuffer(ByteBuffer byteBuffer, int scale)
public static org.apache.avro.Schema getSchemaFor(String str)
public static org.apache.avro.Schema getSchemaFor(File file)
public static org.apache.avro.Schema getSchemaFor(InputStream stream)
public static org.apache.avro.Schema getSchemaFor(URL url)
public static int getIntFromSchema(org.apache.avro.Schema schema, String name)
public static void handleAlterTableForAvro(HiveConf conf, String serializationLib, Map<String,String> parameters)
conf
- serializationLib
- parameters
- Copyright © 2022 The Apache Software Foundation. All rights reserved.