org.apache.hive.hcatalog.listener

Class NotificationListener

java.lang.Object

org.apache.hadoop.hive.metastore.MetaStoreEventListener

org.apache.hive.hcatalog.listener.NotificationListener

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class NotificationListener
extends MetaStoreEventListener

Implementation of MetaStoreEventListener It sends message on two type of topics. One has name of form dbName.tblName On this topic, two kind of messages are sent: add/drop partition and finalize_partition message. Second topic has name "HCAT" and messages sent on it are: add/drop database and add/drop table. All messages also has a property named "HCAT_EVENT" set on them whose value can be used to configure message selector on subscriber side.

Field Summary

Fields

Modifier and Type	Field and Description
protected javax.jms.Connection	conn
static int	NUM_RETRIES
protected ThreadLocal<javax.jms.Session>	session

Constructor Summary

Constructors

Constructor and Description
NotificationListener(org.apache.hadoop.conf.Configuration conf) Create message bus connection and session in constructor.

Method Summary

Modifier and Type	Method and Description
protected javax.jms.Connection	createConnection() Create the JMS connection
protected javax.jms.MessageProducer	createProducer(javax.jms.Destination topic) Create a JMS producer
protected javax.jms.Session	createSession() Creates a JMS session
protected javax.jms.Topic	createTopic(String topicName) Get the topic object for the topicName
protected void	finalize()
protected boolean	isConnectionHealthy() Send a dummy message to probe if the JMS connection is healthy
void	onAddPartition(AddPartitionEvent partitionEvent)
void	onAlterPartition(AlterPartitionEvent ape)
void	onAlterTable(AlterTableEvent tableEvent) Send altered table notifications.
void	onCreateDatabase(CreateDatabaseEvent dbEvent)
void	onCreateTable(CreateTableEvent tableEvent)
void	onDropDatabase(DropDatabaseEvent dbEvent)
void	onDropPartition(DropPartitionEvent partitionEvent) Send dropped partition notifications.
void	onDropTable(DropTableEvent tableEvent) Send dropped table notifications.
void	onLoadPartitionDone(LoadPartitionDoneEvent lpde)
protected void	send(HCatEventMessage hCatEventMessage, String topicName)
protected void	send(HCatEventMessage hCatEventMessage, String topicName, int retries)
protected void	testAndCreateConnection() Does a health check on the connection by sending a dummy message.

Methods inherited from class org.apache.hadoop.hive.metastore.MetaStoreEventListener

getConf, onAddIndex, onAlterIndex, onConfigChange, onDropIndex, onInsert, setConf

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conn

protected javax.jms.Connection conn

NUM_RETRIES

public static final int NUM_RETRIES

See Also:

Constant Field Values

session

protected final ThreadLocal<javax.jms.Session> session

Constructor Detail

NotificationListener

public NotificationListener(org.apache.hadoop.conf.Configuration conf)

Create message bus connection and session in constructor.

Method Detail

onAddPartition

public void onAddPartition(AddPartitionEvent partitionEvent)
                    throws MetaException

Overrides:

onAddPartition in class MetaStoreEventListener

Throws:

MetaException

onAlterPartition

public void onAlterPartition(AlterPartitionEvent ape)
                      throws MetaException

Overrides:

onAlterPartition in class MetaStoreEventListener

Parameters:

ape - partition event

Throws:

MetaException

onDropPartition

public void onDropPartition(DropPartitionEvent partitionEvent)
                     throws MetaException

Send dropped partition notifications. Subscribers can receive these notifications for a particular table by listening on a topic named "dbName.tableName" with message selector string "HCAT_EVENT" = "DROP_PARTITION".
TODO: DataNucleus 2.0.3, currently used by the HiveMetaStore for persistence, has been found to throw NPE when serializing objects that contain null. For this reason we override some fields in the StorageDescriptor of this notification. This should be fixed after HIVE-2084 "Upgrade datanucleus from 2.0.3 to 3.0.1" is resolved.

Overrides:

onDropPartition in class MetaStoreEventListener

Parameters:

partitionEvent - partition event

Throws:

MetaException

onCreateDatabase

public void onCreateDatabase(CreateDatabaseEvent dbEvent)
                      throws MetaException

Overrides:

onCreateDatabase in class MetaStoreEventListener

Parameters:

dbEvent - database event

Throws:

MetaException

onDropDatabase

public void onDropDatabase(DropDatabaseEvent dbEvent)
                    throws MetaException

Overrides:

onDropDatabase in class MetaStoreEventListener

Parameters:

dbEvent - database event

Throws:

MetaException

onCreateTable

public void onCreateTable(CreateTableEvent tableEvent)
                   throws MetaException

Overrides:

onCreateTable in class MetaStoreEventListener

Parameters:

tableEvent - table event.

Throws:

MetaException

onAlterTable

public void onAlterTable(AlterTableEvent tableEvent)
                  throws MetaException

Send altered table notifications. Subscribers can receive these notifications for dropped tables by listening on topic "HCAT" with message selector string "HCAT_EVENT" = "ALTER_TABLE"

Overrides:

onAlterTable in class MetaStoreEventListener

Parameters:

tableEvent - alter table event

Throws:

MetaException

onDropTable

public void onDropTable(DropTableEvent tableEvent)
                 throws MetaException

Send dropped table notifications. Subscribers can receive these notifications for dropped tables by listening on topic "HCAT" with message selector string "HCAT_EVENT" = "DROP_TABLE"
TODO: DataNucleus 2.0.3, currently used by the HiveMetaStore for persistence, has been found to throw NPE when serializing objects that contain null. For this reason we override some fields in the StorageDescriptor of this notification. This should be fixed after HIVE-2084 "Upgrade datanucleus from 2.0.3 to 3.0.1" is resolved.

Overrides:

onDropTable in class MetaStoreEventListener

Parameters:

tableEvent - table event.

Throws:

MetaException

send

protected void send(HCatEventMessage hCatEventMessage,
                    String topicName)

Parameters:

hCatEventMessage - The HCatEventMessage being sent over JMS.

topicName - is the name on message broker on which message is sent.

send

protected void send(HCatEventMessage hCatEventMessage,
                    String topicName,
                    int retries)

Parameters:

hCatEventMessage - The HCatEventMessage being sent over JMS, this method is threadsafe

topicName - is the name on message broker on which message is sent.

retries - the number of retry attempts

createTopic

protected javax.jms.Topic createTopic(String topicName)
                               throws javax.jms.JMSException

Get the topic object for the topicName

Parameters:

topicName - The String identifying the message-topic.

Returns:

A Topic object corresponding to the specified topicName.

Throws:

javax.jms.JMSException

testAndCreateConnection

protected void testAndCreateConnection()

Does a health check on the connection by sending a dummy message. Create the connection if the connection is found to be bad Also recreates the session

createConnection

protected javax.jms.Connection createConnection()

Create the JMS connection

Returns:

newly created JMS connection

isConnectionHealthy

protected boolean isConnectionHealthy()

Send a dummy message to probe if the JMS connection is healthy

Returns:

true if connection is healthy, false otherwise

createSession

protected javax.jms.Session createSession()
                                   throws javax.jms.JMSException

Creates a JMS session

Returns:

newly create JMS session

Throws:

javax.jms.JMSException

createProducer

protected javax.jms.MessageProducer createProducer(javax.jms.Destination topic)
                                            throws javax.jms.JMSException

Create a JMS producer

Parameters:

topic -

Returns:

newly created message producer

Throws:

javax.jms.JMSException

finalize

protected void finalize()
                 throws Throwable

Overrides:

finalize in class Object

Throws:

Throwable

onLoadPartitionDone

public void onLoadPartitionDone(LoadPartitionDoneEvent lpde)
                         throws MetaException

Overrides:

onLoadPartitionDone in class MetaStoreEventListener

Throws:

MetaException