public class DbNotificationListener extends MetaStoreEventListener
This listener takes a ListenerEvent, builds a NotificationEvent, and stores it in the database.This class can be configured as a listener (transaction or non-transactional) so that the listener events are logged into the database.
Design overview: This class implementsMetaStoreEventListener
When a NotificationEvent is created, an unique and monotonically
increasing EVENT_ID is generated from the database and it assigned to each NotificationEvent.
It is important to note that this Listener can be configured as a transaction listener
in cases where client applications rely on the fact that the event is generated only when
the metadata transaction is successful. Each NotificationEvent requires a EVENT_ID which is generated
using a RW row lock in the database in the method RawStore.addNotificationEvent. This means any
other concurrent transaction thread trying to create a event at the same time will block until the
lock is released. This is required to satisfy the strict constraints (monotonically increasing event id,
with no holes and generated only when transaction successful) on the client side
Also, there is a cleaner thread which deletes old notification events at a regular interval. The time-to-live for the Notification Events can be configured by setting hive.metastore.event.db.listener.timetolive appropriately
TODO : The code to generate EVENT_ID gets a database R/W row lock using SELECT FOR UPDATE for the single row on the NOTIFICATION_SEQUENCE table which blocks all the other metadata transactions until the event is committed along with parent transaction. This is likely to cause performance problem and the design needs to be improved
Constructor and Description |
---|
DbNotificationListener(org.apache.hadoop.conf.Configuration config) |
Modifier and Type | Method and Description |
---|---|
void |
onAddIndex(AddIndexEvent indexEvent) |
void |
onAddPartition(AddPartitionEvent partitionEvent) |
void |
onAlterIndex(AlterIndexEvent indexEvent) |
void |
onAlterPartition(AlterPartitionEvent partitionEvent) |
void |
onAlterTable(AlterTableEvent tableEvent) |
void |
onConfigChange(ConfigChangeEvent tableEvent) |
void |
onCreateDatabase(CreateDatabaseEvent dbEvent) |
void |
onCreateFunction(CreateFunctionEvent fnEvent) |
void |
onCreateTable(CreateTableEvent tableEvent) |
void |
onDropDatabase(DropDatabaseEvent dbEvent) |
void |
onDropFunction(DropFunctionEvent fnEvent) |
void |
onDropIndex(DropIndexEvent indexEvent) |
void |
onDropPartition(DropPartitionEvent partitionEvent) |
void |
onDropTable(DropTableEvent tableEvent) |
void |
onInsert(InsertEvent insertEvent)
This will be called when an insert is executed that does not cause a partition to be added.
|
void |
onLoadPartitionDone(LoadPartitionDoneEvent partSetDoneEvent) |
getConf, setConf
public DbNotificationListener(org.apache.hadoop.conf.Configuration config) throws MetaException
MetaException
public void onConfigChange(ConfigChangeEvent tableEvent) throws MetaException
onConfigChange
in class MetaStoreEventListener
tableEvent
- table event.MetaException
public void onCreateTable(CreateTableEvent tableEvent) throws MetaException
onCreateTable
in class MetaStoreEventListener
tableEvent
- table event.MetaException
public void onDropTable(DropTableEvent tableEvent) throws MetaException
onDropTable
in class MetaStoreEventListener
tableEvent
- table event.MetaException
public void onAlterTable(AlterTableEvent tableEvent) throws MetaException
onAlterTable
in class MetaStoreEventListener
tableEvent
- alter table eventMetaException
public void onAddPartition(AddPartitionEvent partitionEvent) throws MetaException
onAddPartition
in class MetaStoreEventListener
partitionEvent
- partition eventMetaException
public void onDropPartition(DropPartitionEvent partitionEvent) throws MetaException
onDropPartition
in class MetaStoreEventListener
partitionEvent
- partition eventMetaException
public void onAlterPartition(AlterPartitionEvent partitionEvent) throws MetaException
onAlterPartition
in class MetaStoreEventListener
partitionEvent
- partition eventMetaException
public void onCreateDatabase(CreateDatabaseEvent dbEvent) throws MetaException
onCreateDatabase
in class MetaStoreEventListener
dbEvent
- database eventMetaException
public void onDropDatabase(DropDatabaseEvent dbEvent) throws MetaException
onDropDatabase
in class MetaStoreEventListener
dbEvent
- database eventMetaException
public void onCreateFunction(CreateFunctionEvent fnEvent) throws MetaException
onCreateFunction
in class MetaStoreEventListener
fnEvent
- function eventMetaException
public void onDropFunction(DropFunctionEvent fnEvent) throws MetaException
onDropFunction
in class MetaStoreEventListener
fnEvent
- function eventMetaException
public void onAddIndex(AddIndexEvent indexEvent) throws MetaException
onAddIndex
in class MetaStoreEventListener
indexEvent
- index eventMetaException
public void onDropIndex(DropIndexEvent indexEvent) throws MetaException
onDropIndex
in class MetaStoreEventListener
indexEvent
- index eventMetaException
public void onAlterIndex(AlterIndexEvent indexEvent) throws MetaException
onAlterIndex
in class MetaStoreEventListener
indexEvent
- index eventMetaException
public void onInsert(InsertEvent insertEvent) throws MetaException
MetaStoreEventListener
MetaStoreEventListener.onAddPartition(org.apache.hadoop.hive.metastore.events.AddPartitionEvent)
to be
called instead.onInsert
in class MetaStoreEventListener
MetaException
public void onLoadPartitionDone(LoadPartitionDoneEvent partSetDoneEvent) throws MetaException
onLoadPartitionDone
in class MetaStoreEventListener
partSetDoneEvent
- MetaException
Copyright © 2021 The Apache Software Foundation. All rights reserved.