public final class DbTxnManager extends Object
DbTxnManager
per SessionState
with a single thread accessing it at a time, with the exception of heartbeat()
method.
The later may (usually will) be called from a timer thread.
See getMS()
for more important concurrency/metastore access notes.
Each statement that the TM (transaction manager) should be aware of should belong to a transaction.
Effectively, that means any statement that has side effects. Exceptions are statements like
Show Compactions, Show Tables, Use Database foo, etc. The transaction is started either
explicitly ( via Start Transaction SQL statement from end user - not fully supported) or
implicitly by the Driver
(which looks exactly as autoCommit=true
from end user poit of view). See more at isExplicitTransaction
.Modifier and Type | Class and Description |
---|---|
static class |
DbTxnManager.Heartbeater
Heartbeater thread
|
static class |
DbTxnManager.HeartbeaterThread |
Modifier and Type | Method and Description |
---|---|
void |
acquireLocks(QueryPlan plan,
Context ctx,
String username)
Acquire all of the locks needed by a query.
|
void |
acquireLocks(QueryPlan plan,
Context ctx,
String username,
Driver.LockedDriverState lDrvState)
Acquire all of the locks needed by a query.
|
LockResponse |
acquireMaterializationRebuildLock(String dbName,
String tableName,
long txnId)
Acquire the materialization rebuild lock for a given view.
|
void |
closeTxnManager()
This call closes down the transaction manager.
|
void |
commitTxn()
Commit the current transaction.
|
protected void |
destruct() |
protected void |
finalize() |
protected HiveLockManager |
getAndCheckLockManager()
Gets the lock manager and verifies if the explicit lock is supported
|
org.apache.hadoop.conf.Configuration |
getConf() |
long |
getCurrentTxnId()
if
isTxnOpen() , returns the currently active transaction ID. |
static long |
getHeartbeatInterval(org.apache.hadoop.conf.Configuration conf) |
HiveLockManager |
getLockManager()
we don't expect multiple threads to call this method concurrently but
lockMgr will
be read by a different threads than one writing it, thus it's volatile |
int |
getStmtIdAndIncrement()
Should be though of more as a unique write operation ID in a given txn (at QueryPlan level).
|
long |
getTableWriteId(String dbName,
String tableName)
if
isTxnOpen() , returns the table write ID associated with current active transaction. |
String |
getTxnManagerName()
Get the name for currently installed transaction manager.
|
org.apache.hadoop.hive.common.ValidTxnList |
getValidTxns()
Get the transactions that are currently valid.
|
org.apache.hadoop.hive.common.ValidTxnWriteIdList |
getValidWriteIds(List<String> tableList,
String validTxnList)
Get the table write Ids that are valid for the current transaction.
|
void |
heartbeat()
Send a heartbeat to the transaction management storage so other Hive
clients know that the transaction and locks held by this client are
still valid.
|
boolean |
isImplicitTransactionOpen() |
boolean |
isTxnOpen() |
int |
lockDatabase(Hive hiveDB,
LockDatabaseDesc lockDb)
This function is called to lock the database when explicit lock command is
issued on a database.
|
int |
lockTable(Hive db,
LockTableDesc lockTbl)
This function is called to lock the table when explicit lock command is
issued on a table.
|
long |
openTxn(Context ctx,
String user)
Open a new transaction.
|
boolean |
recordSnapshot(QueryPlan queryPlan)
In an explicit txn start_transaction is the 1st statement and we record the snapshot at the
start of the txn for Snapshot Isolation.
|
void |
releaseLocks(List<HiveLock> hiveLocks)
Release specified locks.
|
void |
replAllocateTableWriteIdsBatch(String dbName,
String tableName,
String replPolicy,
List<TxnToWriteId> srcTxnToWriteIdList)
Allocates write id for each transaction in the list.
|
void |
replCommitTxn(String replPolicy,
long srcTxnId)
Commit the transaction in target cluster.
|
List<Long> |
replOpenTxn(String replPolicy,
List<Long> srcTxnIds,
String user)
Open a new transaction in target cluster.
|
void |
replRollbackTxn(String replPolicy,
long srcTxnId)
Abort the transaction in target cluster.
|
void |
replTableWriteIdState(String validWriteIdList,
String dbName,
String tableName,
List<String> partNames)
Replicate Table Write Ids state to mark aborted write ids and writeid high water mark.
|
void |
rollbackTxn()
Abort the current transaction.
|
void |
setConf(org.apache.hadoop.conf.Configuration c) |
boolean |
supportsAcid()
Indicate whether this transaction manager supports ACID operations.
|
boolean |
supportsExplicitLock()
Indicate whether this lock manager supports the use of
lock
database or lock table . |
int |
unlockDatabase(Hive hiveDB,
UnlockDatabaseDesc unlockDb)
This function is called to unlock the database when explicit unlock command
is issued on a database.
|
int |
unlockTable(Hive hiveDB,
UnlockTableDesc unlockTbl)
This function is called to unlock the table when explicit unlock command is
issued on a table.
|
boolean |
useNewShowLocksFormat()
Indicate whether this transaction manager returns information about locks in the new format
for show locks or the old one.
|
protected HiveConf conf
public List<Long> replOpenTxn(String replPolicy, List<Long> srcTxnIds, String user) throws LockException
HiveTxnManager
replPolicy
- Replication policy to uniquely identify the source cluster.srcTxnIds
- The ids of the transaction at the source clusteruser
- The user who has fired the repl load commandLockException
- in case of failure to start the transaction.public long openTxn(Context ctx, String user) throws LockException
HiveTxnManager
ctx
- Context for this queryuser
- Hive user who is opening this transaction.LockException
- if a transaction is already open.public HiveLockManager getLockManager() throws LockException
lockMgr
will
be read by a different threads than one writing it, thus it's volatile
LockException
- if there is an issue obtaining the lock manager.public void acquireLocks(QueryPlan plan, Context ctx, String username) throws LockException
HiveTxnManager
HiveTxnManager.openTxn(Context, String)
.
A list of acquired locks will be stored in the
Context
object and can be retrieved
via Context.getHiveLocks()
.plan
- query planctx
- Context for this queryusername
- name of the user for this queryLockException
- if there is an error getting the locks. Use HiveException.getCanonicalErrorMsg()
to get more info on how to handle the exception.public void releaseLocks(List<HiveLock> hiveLocks) throws LockException
HiveTxnManager
supportsAcid() == true
,
will track locks internally and ignore this parameterhiveLocks
- The list of locks to be released.LockException
public void replCommitTxn(String replPolicy, long srcTxnId) throws LockException
HiveTxnManager
replPolicy
- Replication policy to uniquely identify the source cluster.srcTxnId
- The id of the transaction at the source clusterLockException
- in case of failure to commit the transaction.public void commitTxn() throws LockException
HiveTxnManager
HiveTxnManager.acquireLocks(org.apache.hadoop.hive.ql.QueryPlan,
org.apache.hadoop.hive.ql.Context, java.lang.String)
.LockException
- if there is no current transaction or the
transaction has already been committed or aborted.public void replRollbackTxn(String replPolicy, long srcTxnId) throws LockException
HiveTxnManager
replPolicy
- Replication policy to uniquely identify the source cluster.srcTxnId
- The id of the transaction at the source clusterLockException
- in case of failure to abort the transaction.public void rollbackTxn() throws LockException
HiveTxnManager
HiveTxnManager.acquireLocks(org.apache.hadoop.hive.ql.QueryPlan,
org.apache.hadoop.hive.ql.Context, java.lang.String)
.LockException
- if there is no current transaction or the
transaction has already been committed or aborted.public void replTableWriteIdState(String validWriteIdList, String dbName, String tableName, List<String> partNames) throws LockException
HiveTxnManager
validWriteIdList
- Snapshot of writeid list when the table/partition is dumped.dbName
- Database nametableName
- Table which is written.partNames
- List of partitions being written.LockException
- in case of failure.public void heartbeat() throws LockException
HiveTxnManager
LockException
- If current transaction exists or the transaction
has already been committed or aborted.public org.apache.hadoop.hive.common.ValidTxnList getValidTxns() throws LockException
HiveTxnManager
ValidTxnList
object can be passed as string to the processing
tasks for use in the reading the data. This call should be made once up
front by the planner and should never be called on the backend,
as this will violate the isolation level semantics.LockException
public org.apache.hadoop.hive.common.ValidTxnWriteIdList getValidWriteIds(List<String> tableList, String validTxnList) throws LockException
HiveTxnManager
ValidTxnWriteIdList
object can be passed as string to the processing
tasks for use in the reading the data. This call will return same results as long as validTxnString
passed is same.tableList
- list of tables (validTxnList
- snapshot of valid txns for the current txnLockException
public String getTxnManagerName()
HiveTxnManager
public boolean supportsExplicitLock()
HiveTxnManager
lock
database
or lock table
.public int lockTable(Hive db, LockTableDesc lockTbl) throws HiveException
HiveTxnManager
lockTable
in interface HiveTxnManager
db
- an object to communicate with the metastorelockTbl
- table locking info, such as table name, locking modeHiveException
public int unlockTable(Hive hiveDB, UnlockTableDesc unlockTbl) throws HiveException
HiveTxnManager
unlockTable
in interface HiveTxnManager
hiveDB
- an object to communicate with the metastoreunlockTbl
- table unlocking info, such as table nameHiveException
public int lockDatabase(Hive hiveDB, LockDatabaseDesc lockDb) throws HiveException
HiveTxnManager
lockDatabase
in interface HiveTxnManager
hiveDB
- an object to communicate with the metastorelockDb
- database locking info, such as database name, locking modeHiveException
public int unlockDatabase(Hive hiveDB, UnlockDatabaseDesc unlockDb) throws HiveException
HiveTxnManager
unlockDatabase
in interface HiveTxnManager
hiveDB
- an object to communicate with the metastoreunlockDb
- database unlocking info, such as database nameHiveException
public boolean useNewShowLocksFormat()
HiveTxnManager
public boolean supportsAcid()
HiveTxnManager
public boolean recordSnapshot(QueryPlan queryPlan)
recordSnapshot
in interface HiveTxnManager
public boolean isImplicitTransactionOpen()
isImplicitTransactionOpen
in interface HiveTxnManager
protected void destruct()
public boolean isTxnOpen()
public long getCurrentTxnId()
HiveTxnManager
isTxnOpen()
, returns the currently active transaction ID.public int getStmtIdAndIncrement()
HiveTxnManager
public long getTableWriteId(String dbName, String tableName) throws LockException
HiveTxnManager
isTxnOpen()
, returns the table write ID associated with current active transaction.LockException
public LockResponse acquireMaterializationRebuildLock(String dbName, String tableName, long txnId) throws LockException
HiveTxnManager
acquireMaterializationRebuildLock
in interface HiveTxnManager
LockException
public void replAllocateTableWriteIdsBatch(String dbName, String tableName, String replPolicy, List<TxnToWriteId> srcTxnToWriteIdList) throws LockException
HiveTxnManager
dbName
- database nametableName
- the name of the table to allocate the write idreplPolicy
- used by replication task to identify the source clustersrcTxnToWriteIdList
- List of txn id to write id MapLockException
public static long getHeartbeatInterval(org.apache.hadoop.conf.Configuration conf) throws LockException
LockException
public void setConf(org.apache.hadoop.conf.Configuration c)
setConf
in interface org.apache.hadoop.conf.Configurable
public org.apache.hadoop.conf.Configuration getConf()
getConf
in interface org.apache.hadoop.conf.Configurable
public void closeTxnManager()
HiveTxnManager
closeTxnManager
in interface HiveTxnManager
public void acquireLocks(QueryPlan plan, Context ctx, String username, Driver.LockedDriverState lDrvState) throws LockException
HiveTxnManager
HiveTxnManager.openTxn(Context, String)
.
A list of acquired locks will be stored in the
Context
object and can be retrieved
via Context.getHiveLocks()
.acquireLocks
in interface HiveTxnManager
plan
- query planctx
- Context for this queryusername
- name of the user for this querylDrvState
- the state to inform if the query cancelled or notLockException
- if there is an error getting the locksprotected void finalize() throws Throwable
protected HiveLockManager getAndCheckLockManager() throws HiveException
HiveException
Copyright © 2022 The Apache Software Foundation. All rights reserved.