org.apache.hive.hcatalog.api

Class HCatClient

java.lang.Object

org.apache.hive.hcatalog.api.HCatClient

Direct Known Subclasses:

HCatClientHMSImpl

public abstract class HCatClient
extends Object

The abstract class HCatClient containing APIs for HCatalog DDL commands.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HCatClient.DropDBMode

Field Summary

Fields

Modifier and Type	Field and Description
static String	HCAT_CLIENT_IMPL_CLASS

Constructor Summary

Constructors

Constructor and Description
HCatClient()

Method Summary

Methods

Modifier and Type	Method and Description
abstract void	addPartition(HCatAddPartitionDesc partInfo) Adds the partition.
abstract int	addPartitions(List<HCatAddPartitionDesc> partInfoList) Adds a list of partitions.
abstract int	addPartitionSpec(HCatPartitionSpec partitionSpec) Adds partitions using HCatPartitionSpec.
abstract void	cancelDelegationToken(String tokenStrForm) Cancel delegation token.
abstract void	close() Close the hcatalog client.
static HCatClient	create(org.apache.hadoop.conf.Configuration conf) Creates an instance of HCatClient.
abstract void	createDatabase(HCatCreateDBDesc dbInfo) Creates the database.
abstract void	createTable(HCatCreateTableDesc createTableDesc) Creates the table.
abstract void	createTableLike(String dbName, String existingTblName, String newTableName, boolean ifNotExists, boolean isExternal, String location) Creates the table like an existing table.
abstract HCatPartition	deserializePartition(String hcatPartitionStringRep) Deserializer for an HCatPartition.
abstract List<HCatPartition>	deserializePartitions(List<String> hcatPartitionStringReps) Deserializer for a list of HCatPartition strings.
abstract HCatPartitionSpec	deserializePartitionSpec(List<String> hcatPartitionSpecStrings) Deserializer for HCatPartitionSpec.
abstract HCatTable	deserializeTable(String hcatTableStringRep) Deserializer for HCatTable.
abstract void	dropDatabase(String dbName, boolean ifExists, HCatClient.DropDBMode mode) Drops a database.
abstract void	dropPartitions(String dbName, String tableName, Map<String,String> partitionSpec, boolean ifExists) Drops partition(s) that match the specified (and possibly partial) partition specification.
abstract void	dropPartitions(String dbName, String tableName, Map<String,String> partitionSpec, boolean ifExists, boolean deleteData) Drops partition(s) that match the specified (and possibly partial) partition specification.
abstract void	dropTable(String dbName, String tableName, boolean ifExists) Drop table.
abstract String	getConfVal(String key, String defaultVal) Fetch configuration value on conf that the HCatClient is instantiated against.
abstract long	getCurrentNotificationEventId() Get the most recently used notification id.
abstract HCatDatabase	getDatabase(String dbName) Gets the database.
abstract String	getDelegationToken(String owner, String renewerKerberosPrincipalName) Gets the delegation token.
abstract String	getMessageBusTopicName(String dbName, String tableName) Retrieve Message-bus topic for a table.
abstract List<HCatNotificationEvent>	getNextNotification(long lastEventId, int maxEvents, IMetaStoreClient.NotificationFilter filter) Get a list of notifications
abstract HCatPartition	getPartition(String dbName, String tableName, Map<String,String> partitionSpec) Gets the partition.
abstract List<HCatPartition>	getPartitions(String dbName, String tblName) Gets all the partitions.
abstract List<HCatPartition>	getPartitions(String dbName, String tblName, Map<String,String> partitionSpec) Gets all the partitions that match the specified (and possibly partial) partition specification.
abstract HCatPartitionSpec	getPartitionSpecs(String dbName, String tableName, int maxPartitions) Gets partitions in terms of generic HCatPartitionSpec instances.
abstract HCatPartitionSpec	getPartitionSpecs(String dbName, String tableName, Map<String,String> partitionSelector, int maxPartitions) Gets partitions in terms of generic HCatPartitionSpec instances.
abstract Iterator<ReplicationTask>	getReplicationTasks(long lastEventId, int maxEvents, String dbName, String tableName) Get an iterator that iterates over a list of replication tasks needed to replicate all the events that have taken place for a given db/table.
abstract HCatTable	getTable(String dbName, String tableName) Gets the table.
abstract boolean	isPartitionMarkedForEvent(String dbName, String tblName, Map<String,String> partKVs, PartitionEventType eventType) Checks if a partition is marked for event.
abstract List<String>	listDatabaseNamesByPattern(String pattern) Get all existing databases that match the given pattern.
abstract List<HCatPartition>	listPartitionsByFilter(String dbName, String tblName, String filter) List partitions by filter.
abstract HCatPartitionSpec	listPartitionSpecsByFilter(String dbName, String tblName, String filter, int maxPartitions) List partitions by filter, but as HCatPartitionSpecs.
abstract List<String>	listTableNamesByPattern(String dbName, String tablePattern) Returns all existing tables from the specified database which match the given pattern.
abstract void	markPartitionForEvent(String dbName, String tblName, Map<String,String> partKVs, PartitionEventType eventType) Mark partition for event.
abstract void	renameTable(String dbName, String oldName, String newName) Renames a table.
abstract long	renewDelegationToken(String tokenStrForm) Renew delegation token.
abstract String	serializePartition(HCatPartition hcatPartition) Serializer for HCatPartition.
abstract List<String>	serializePartitions(List<HCatPartition> hcatPartitions) Serializer for a list of HCatPartition.
abstract List<String>	serializePartitionSpec(HCatPartitionSpec partitionSpec) Serializer for HCatPartitionSpec.
abstract String	serializeTable(HCatTable hcatTable) Serializer for HCatTable.
abstract void	updateTableSchema(String dbName, String tableName, HCatTable newTableDefinition) Updates the Table's whole schema (including column schema, I/O Formats, SerDe definitions, etc.)
abstract void	updateTableSchema(String dbName, String tableName, List<HCatFieldSchema> columnSchema) Updates the Table's column schema to the specified definition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HCAT_CLIENT_IMPL_CLASS

public static final String HCAT_CLIENT_IMPL_CLASS

See Also:

Constant Field Values

Constructor Detail

HCatClient

public HCatClient()

Method Detail

create

public static HCatClient create(org.apache.hadoop.conf.Configuration conf)
                         throws HCatException

Creates an instance of HCatClient.

Parameters:

conf - An instance of configuration.

Returns:

An instance of HCatClient.

Throws:

HCatException

getConfVal

public abstract String getConfVal(String key,
                String defaultVal)

Fetch configuration value on conf that the HCatClient is instantiated against. We do not want to expose the conf itself via a getConf(), because we do not want it modifiable after instantiation of the HCatClient, but modules that get called from HCatClient often need to know about how HCatClient is configured, so we want a read-only interface for it.

Parameters:

key - keyname to look up

defaultVal - default value to furnish in case the key does not exist

Returns:

value for given key, and defaultVal if key is not present in conf

listDatabaseNamesByPattern

public abstract List<String> listDatabaseNamesByPattern(String pattern)
                                                 throws HCatException

Get all existing databases that match the given pattern. The matching occurs as per Java regular expressions

Parameters:

pattern - java re pattern

Returns:

list of database names

Throws:

HCatException

getDatabase

public abstract HCatDatabase getDatabase(String dbName)
                                  throws HCatException

Gets the database.

Parameters:

dbName - The name of the database.

Returns:

An instance of HCatDatabaseInfo.

Throws:

HCatException

createDatabase

public abstract void createDatabase(HCatCreateDBDesc dbInfo)
                             throws HCatException

Creates the database.

Parameters:

dbInfo - An instance of HCatCreateDBDesc.

Throws:

HCatException

dropDatabase

public abstract void dropDatabase(String dbName,
                boolean ifExists,
                HCatClient.DropDBMode mode)
                           throws HCatException

Drops a database.

Parameters:

dbName - The name of the database to delete.

ifExists - Hive returns an error if the database specified does not exist, unless ifExists is set to true.

mode - This is set to either "restrict" or "cascade". Restrict will remove the schema if all the tables are empty. Cascade removes everything including data and definitions.

Throws:

HCatException

listTableNamesByPattern

public abstract List<String> listTableNamesByPattern(String dbName,
                                   String tablePattern)
                                              throws HCatException

Returns all existing tables from the specified database which match the given pattern. The matching occurs as per Java regular expressions.

Parameters:

dbName - The name of the DB (to be searched)

tablePattern - The regex for the table-name

Returns:

list of table names

Throws:

HCatException

getTable

public abstract HCatTable getTable(String dbName,
                 String tableName)
                            throws HCatException

Gets the table.

Parameters:

dbName - The name of the database.

tableName - The name of the table.

Returns:

An instance of HCatTableInfo.

Throws:

HCatException

createTable

public abstract void createTable(HCatCreateTableDesc createTableDesc)
                          throws HCatException

Creates the table.

Parameters:

createTableDesc - An instance of HCatCreateTableDesc class.

Throws:

HCatException

updateTableSchema

public abstract void updateTableSchema(String dbName,
                     String tableName,
                     List<HCatFieldSchema> columnSchema)
                                throws HCatException

Updates the Table's column schema to the specified definition.

Parameters:

dbName - The name of the database.

tableName - The name of the table.

columnSchema - The (new) definition of the column schema (i.e. list of fields).

Throws:

HCatException

updateTableSchema

public abstract void updateTableSchema(String dbName,
                     String tableName,
                     HCatTable newTableDefinition)
                                throws HCatException

Updates the Table's whole schema (including column schema, I/O Formats, SerDe definitions, etc.)

Parameters:

dbName - The name of the database.

tableName - The name of the table.

newTableDefinition - The (new) definition of the table.

Throws:

HCatException

serializeTable

public abstract String serializeTable(HCatTable hcatTable)
                               throws HCatException

Serializer for HCatTable.

Parameters:

hcatTable - The HCatTable to be serialized into string form

Returns:

String representation of the HCatTable.

Throws:

HCatException, - on failure to serialize.

HCatException

deserializeTable

public abstract HCatTable deserializeTable(String hcatTableStringRep)
                                    throws HCatException

Deserializer for HCatTable.

Parameters:

hcatTableStringRep - The String representation of an HCatTable, presumably retrieved from serializeTable(HCatTable)

Returns:

HCatTable reconstructed from the string throws HCatException, on failure to deserialize.

Throws:

HCatException

serializePartition

public abstract String serializePartition(HCatPartition hcatPartition)
                                   throws HCatException

Serializer for HCatPartition.

Parameters:

hcatPartition - The HCatPartition instance to be serialized.

Returns:

String representation of the HCatPartition.

Throws:

HCatException, - on failure to serialize.

HCatException

serializePartitions

public abstract List<String> serializePartitions(List<HCatPartition> hcatPartitions)
                                          throws HCatException

Serializer for a list of HCatPartition.

Parameters:

hcatPartitions - The HCatPartitions to be serialized.

Returns:

A list of Strings, each representing an HCatPartition.

Throws:

HCatException, - on failure to serialize.

HCatException

deserializePartition

public abstract HCatPartition deserializePartition(String hcatPartitionStringRep)
                                            throws HCatException

Deserializer for an HCatPartition.

Parameters:

hcatPartitionStringRep - The String representation of the HCatPartition, presumably retrieved from serializePartition(HCatPartition)

Returns:

HCatPartition instance reconstructed from the string.

Throws:

HCatException, - on failure to deserialze.

HCatException

deserializePartitions

public abstract List<HCatPartition> deserializePartitions(List<String> hcatPartitionStringReps)
                                                   throws HCatException

Deserializer for a list of HCatPartition strings.

Parameters:

hcatPartitionStringReps - The list of HCatPartition strings to be deserialized.

Returns:

A list of HCatPartition instances, each reconstructed from an entry in the string-list.

Throws:

HCatException, - on failure to deserialize.

HCatException

serializePartitionSpec

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract List<String> serializePartitionSpec(HCatPartitionSpec partitionSpec)
                                             throws HCatException

Serializer for HCatPartitionSpec.

Parameters:

partitionSpec - HCatPartitionSpec to be serialized.

Returns:

A list of Strings, representing the HCatPartitionSpec as a whole.

Throws:

HCatException - On failure to serialize.

deserializePartitionSpec

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract HCatPartitionSpec deserializePartitionSpec(List<String> hcatPartitionSpecStrings)
                                                    throws HCatException

Deserializer for HCatPartitionSpec.

Parameters:

hcatPartitionSpecStrings - List of strings, representing the HCatPartitionSpec as a whole.

Returns:

HCatPartitionSpec, reconstructed from the list of strings.

Throws:

HCatException - On failure to deserialize.

createTableLike

public abstract void createTableLike(String dbName,
                   String existingTblName,
                   String newTableName,
                   boolean ifNotExists,
                   boolean isExternal,
                   String location)
                              throws HCatException

Creates the table like an existing table.

Parameters:

dbName - The name of the database.

existingTblName - The name of the existing table.

newTableName - The name of the new table.

ifNotExists - If true, then error related to already table existing is skipped.

isExternal - Set to "true", if table has be created at a different location other than default.

location - The location for the table.

Throws:

HCatException

dropTable

public abstract void dropTable(String dbName,
             String tableName,
             boolean ifExists)
                        throws HCatException

Drop table.

Parameters:

dbName - The name of the database.

tableName - The name of the table.

ifExists - Hive returns an error if the database specified does not exist, unless ifExists is set to true.

Throws:

HCatException

renameTable

public abstract void renameTable(String dbName,
               String oldName,
               String newName)
                          throws HCatException

Renames a table.

Parameters:

dbName - The name of the database.

oldName - The name of the table to be renamed.

newName - The new name of the table.

Throws:

HCatException

getPartitions

public abstract List<HCatPartition> getPartitions(String dbName,
                                String tblName)
                                           throws HCatException

Gets all the partitions.

Parameters:

dbName - The name of the database.

tblName - The name of the table.

Returns:

A list of partitions.

Throws:

HCatException

getPartitions

public abstract List<HCatPartition> getPartitions(String dbName,
                                String tblName,
                                Map<String,String> partitionSpec)
                                           throws HCatException

Gets all the partitions that match the specified (and possibly partial) partition specification. A partial partition-specification is one where not all partition-keys have associated values. For example, for a table ('myDb.myTable') with 2 partition keys (dt string, region string), if for each dt ('20120101', '20120102', etc.) there can exist 3 regions ('us', 'uk', 'in'), then, 1. Complete partition spec: getPartitions('myDb', 'myTable', {dt='20120101', region='us'}) would return 1 partition. 2. Partial partition spec: getPartitions('myDb', 'myTable', {dt='20120101'}) would return all 3 partitions, with dt='20120101' (i.e. region = 'us', 'uk' and 'in').

Parameters:

dbName - The name of the database.

tblName - The name of the table.

partitionSpec - The partition specification. (Need not include all partition keys.)

Returns:

A list of partitions.

Throws:

HCatException

getPartitionSpecs

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract HCatPartitionSpec getPartitionSpecs(String dbName,
                                                                                                    String tableName,
                                                                                                    int maxPartitions)
                                             throws HCatException

Gets partitions in terms of generic HCatPartitionSpec instances.

Throws:

HCatException

getPartitionSpecs

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract HCatPartitionSpec getPartitionSpecs(String dbName,
                                                                                                    String tableName,
                                                                                                    Map<String,String> partitionSelector,
                                                                                                    int maxPartitions)
                                             throws HCatException

Gets partitions in terms of generic HCatPartitionSpec instances.

Throws:

HCatException

getPartition

public abstract HCatPartition getPartition(String dbName,
                         String tableName,
                         Map<String,String> partitionSpec)
                                    throws HCatException

Gets the partition.

Parameters:

dbName - The database name.

tableName - The table name.

partitionSpec - The partition specification, {[col_name,value],[col_name2,value2]}. All partition-key-values must be specified.

Returns:

An instance of HCatPartitionInfo.

Throws:

HCatException

addPartition

public abstract void addPartition(HCatAddPartitionDesc partInfo)
                           throws HCatException

Adds the partition.

Parameters:

partInfo - An instance of HCatAddPartitionDesc.

Throws:

HCatException

addPartitions

public abstract int addPartitions(List<HCatAddPartitionDesc> partInfoList)
                           throws HCatException

Adds a list of partitions.

Parameters:

partInfoList - A list of HCatAddPartitionDesc.

Returns:

The number of partitions added.

Throws:

HCatException

addPartitionSpec

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract int addPartitionSpec(HCatPartitionSpec partitionSpec)
                              throws HCatException

Adds partitions using HCatPartitionSpec.

Parameters:

partitionSpec - The HCatPartitionSpec representing the set of partitions added.

Returns:

The number of partitions added.

Throws:

HCatException - On failure to add partitions.

dropPartitions

public abstract void dropPartitions(String dbName,
                  String tableName,
                  Map<String,String> partitionSpec,
                  boolean ifExists)
                             throws HCatException

Drops partition(s) that match the specified (and possibly partial) partition specification. A partial partition-specification is one where not all partition-keys have associated values. For example, for a table ('myDb.myTable') with 2 partition keys (dt string, region string), if for each dt ('20120101', '20120102', etc.) there can exist 3 regions ('us', 'uk', 'in'), then, 1. Complete partition spec: dropPartitions('myDb', 'myTable', {dt='20120101', region='us'}) would drop 1 partition. 2. Partial partition spec: dropPartitions('myDb', 'myTable', {dt='20120101'}) would drop all 3 partitions, with dt='20120101' (i.e. region = 'us', 'uk' and 'in').

Parameters:

dbName - The database name.

tableName - The table name.

partitionSpec - The partition specification, {[col_name,value],[col_name2,value2]}.

ifExists - Hive returns an error if the partition specified does not exist, unless ifExists is set to true.

Throws:

HCatException,ConnectionFailureException

HCatException

dropPartitions

public abstract void dropPartitions(String dbName,
                  String tableName,
                  Map<String,String> partitionSpec,
                  boolean ifExists,
                  boolean deleteData)
                             throws HCatException

Drops partition(s) that match the specified (and possibly partial) partition specification. A partial partition-specification is one where not all partition-keys have associated values. For example, for a table ('myDb.myTable') with 2 partition keys (dt string, region string), if for each dt ('20120101', '20120102', etc.) there can exist 3 regions ('us', 'uk', 'in'), then, 1. Complete partition spec: dropPartitions('myDb', 'myTable', {dt='20120101', region='us'}) would drop 1 partition. 2. Partial partition spec: dropPartitions('myDb', 'myTable', {dt='20120101'}) would drop all 3 partitions, with dt='20120101' (i.e. region = 'us', 'uk' and 'in').

Parameters:

dbName - The database name.

tableName - The table name.

partitionSpec - The partition specification, {[col_name,value],[col_name2,value2]}.

ifExists - Hive returns an error if the partition specified does not exist, unless ifExists is set to true.

deleteData - Whether to delete the underlying data.

Throws:

HCatException,ConnectionFailureException

HCatException

listPartitionsByFilter

public abstract List<HCatPartition> listPartitionsByFilter(String dbName,
                                         String tblName,
                                         String filter)
                                                    throws HCatException

List partitions by filter.

Parameters:

dbName - The database name.

tblName - The table name.

filter - The filter string, for example "part1 = \"p1_abc\" and part2 <= "\p2_test\"". Filtering can be done only on string partition keys.

Returns:

list of partitions

Throws:

HCatException

listPartitionSpecsByFilter

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract HCatPartitionSpec listPartitionSpecsByFilter(String dbName,
                                                                                                             String tblName,
                                                                                                             String filter,
                                                                                                             int maxPartitions)
                                                      throws HCatException

List partitions by filter, but as HCatPartitionSpecs.

Throws:

HCatException

markPartitionForEvent

public abstract void markPartitionForEvent(String dbName,
                         String tblName,
                         Map<String,String> partKVs,
                         PartitionEventType eventType)
                                    throws HCatException

Mark partition for event.

Parameters:

dbName - The database name.

tblName - The table name.

partKVs - the key-values associated with the partition.

eventType - the event type

Throws:

HCatException

isPartitionMarkedForEvent

public abstract boolean isPartitionMarkedForEvent(String dbName,
                                String tblName,
                                Map<String,String> partKVs,
                                PartitionEventType eventType)
                                           throws HCatException

Checks if a partition is marked for event.

Parameters:

dbName - the db name

tblName - the table name

partKVs - the key-values associated with the partition.

eventType - the event type

Returns:

true, if is partition marked for event

Throws:

HCatException

getDelegationToken

public abstract String getDelegationToken(String owner,
                        String renewerKerberosPrincipalName)
                                   throws HCatException

Gets the delegation token.

Parameters:

owner - the owner

renewerKerberosPrincipalName - the renewer kerberos principal name

Returns:

the delegation token

Throws:

HCatException,ConnectionFailureException

HCatException

renewDelegationToken

public abstract long renewDelegationToken(String tokenStrForm)
                                   throws HCatException

Renew delegation token.

Parameters:

tokenStrForm - the token string

Returns:

the new expiration time

Throws:

HCatException

cancelDelegationToken

public abstract void cancelDelegationToken(String tokenStrForm)
                                    throws HCatException

Cancel delegation token.

Parameters:

tokenStrForm - the token string

Throws:

HCatException

getMessageBusTopicName

public abstract String getMessageBusTopicName(String dbName,
                            String tableName)
                                       throws HCatException

Retrieve Message-bus topic for a table.

Parameters:

dbName - The name of the DB.

tableName - The name of the table.

Returns:

Topic-name for the message-bus on which messages will be sent for the specified table. By default, this is set to .. Returns null when not set.

Throws:

HCatException

getReplicationTasks

@InterfaceStability.Evolving
public abstract Iterator<ReplicationTask> getReplicationTasks(long lastEventId,
                                                                        int maxEvents,
                                                                        String dbName,
                                                                        String tableName)
                                                       throws HCatException

Get an iterator that iterates over a list of replication tasks needed to replicate all the events that have taken place for a given db/table.

Parameters:

lastEventId - : The last event id that was processed for this reader. The returned replication tasks will start from this point forward

maxEvents - : Maximum number of events to consider for generating the replication tasks. If < 1, then all available events will be considered.

dbName - : The database name for which we're interested in the events for.

tableName - : The table name for which we're interested in the events for - if null, then this function will behave as if it were running at a db level.

Returns:

an iterator over a list of replication events that can be processed one by one.

Throws:

HCatException

getNextNotification

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract List<HCatNotificationEvent> getNextNotification(long lastEventId,
                                                                                                                int maxEvents,
                                                                                                                IMetaStoreClient.NotificationFilter filter)
                                                         throws HCatException

Get a list of notifications

Parameters:

lastEventId - The last event id that was consumed by this reader. The returned notifications will start at the next eventId available this eventId that matches the filter.

maxEvents - Maximum number of events to return. If < 1, then all available events will be returned.

filter - Filter to determine if message should be accepted. If null, then all available events up to maxEvents will be returned.

Returns:

list of notifications, sorted by eventId. It is guaranteed that the events are in the order that the operations were done on the database.

Throws:

HCatException

getCurrentNotificationEventId

@InterfaceAudience.LimitedPrivate(value="Hive")
@InterfaceStability.Evolving
public abstract long getCurrentNotificationEventId()
                                            throws HCatException

Get the most recently used notification id.

Returns:

Throws:

HCatException

close

public abstract void close()
                    throws HCatException

Close the hcatalog client.

Throws:

HCatException