org.apache.hadoop.hive.metastore.txn

Class TxnHandler

java.lang.Object

org.apache.hadoop.hive.metastore.txn.TxnHandler

Direct Known Subclasses:

CompactionTxnHandler

public class TxnHandler
extends Object

A handler to answer transaction related calls that come into the metastore server.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	TxnHandler.DatabaseProduct
protected class	TxnHandler.RetryException

Field Summary

Fields

Modifier and Type	Field and Description
static String	CLEANING_RESPONSE
protected HiveConf	conf
protected TxnHandler.DatabaseProduct	dbProduct
protected int	deadlockCnt Number of consecutive deadlocks we have seen
static String	INITIATED_RESPONSE
protected static char	INITIATED_STATE
protected static char	LOCK_ACQUIRED
protected static char	LOCK_EXCLUSIVE
protected static char	LOCK_SEMI_SHARED
protected static char	LOCK_SHARED
protected static char	LOCK_WAITING
protected static char	MAJOR_TYPE
protected static char	MINOR_TYPE
protected static char	READY_FOR_CLEANING
protected static char	TXN_ABORTED
protected static char	TXN_OPEN
static String	WORKING_RESPONSE
protected static char	WORKING_STATE

Constructor Summary

Constructors

Constructor and Description
TxnHandler(HiveConf conf)

Method Summary

Methods

Modifier and Type	Method and Description
void	abortTxn(AbortTxnRequest rqst)
void	addDynamicPartitions(AddDynamicPartitions rqst)
LockResponse	checkLock(CheckLockRequest rqst)
protected void	checkRetryable(Connection conn, SQLException e, String caller) Determine if an exception was such that it makse sense to retry.
protected void	closeDbConn(Connection dbConn)
protected void	closeStmt(Statement stmt) Close statement instance.
void	commitTxn(CommitTxnRequest rqst)
void	compact(CompactionRequest rqst)
static ValidTxnList	createValidReadTxnList(GetOpenTxnsResponse txns, long currentTxn) Transform a GetOpenTxnsResponse to a ValidTxnList.
protected TxnHandler.DatabaseProduct	determineDatabaseProduct(Connection conn) Determine the database product type
protected Connection	getDbConn(int isolationLevel) Get a connection to the database
protected long	getDbTime(Connection conn) Determine the current time, using the RDBMS as a source of truth
protected String	getIdentifierQuoteString(Connection conn) Determine the String that should be used to quote identifiers.
GetOpenTxnsResponse	getOpenTxns()
GetOpenTxnsInfoResponse	getOpenTxnsInfo()
void	heartbeat(HeartbeatRequest ids)
HeartbeatTxnRangeResponse	heartbeatTxnRange(HeartbeatTxnRangeRequest rqst)
LockResponse	lock(LockRequest rqst)
LockResponse	lockNoWait(LockRequest rqst)
OpenTxnsResponse	openTxns(OpenTxnRequest rqst)
ShowCompactResponse	showCompact(ShowCompactRequest rqst)
ShowLocksResponse	showLocks(ShowLocksRequest rqst)
void	unlock(UnlockRequest rqst)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INITIATED_RESPONSE

public static final String INITIATED_RESPONSE

See Also:

Constant Field Values

WORKING_RESPONSE

public static final String WORKING_RESPONSE

See Also:

Constant Field Values

CLEANING_RESPONSE

public static final String CLEANING_RESPONSE

See Also:

Constant Field Values

INITIATED_STATE

protected static final char INITIATED_STATE

See Also:

Constant Field Values

WORKING_STATE

protected static final char WORKING_STATE

See Also:

Constant Field Values

READY_FOR_CLEANING

protected static final char READY_FOR_CLEANING

See Also:

Constant Field Values

MAJOR_TYPE

protected static final char MAJOR_TYPE

See Also:

Constant Field Values

MINOR_TYPE

protected static final char MINOR_TYPE

See Also:

Constant Field Values

TXN_ABORTED

protected static final char TXN_ABORTED

See Also:

Constant Field Values

TXN_OPEN

protected static final char TXN_OPEN

See Also:

Constant Field Values

LOCK_ACQUIRED

protected static final char LOCK_ACQUIRED

See Also:

Constant Field Values

LOCK_WAITING

protected static final char LOCK_WAITING

See Also:

Constant Field Values

LOCK_EXCLUSIVE

protected static final char LOCK_EXCLUSIVE

See Also:

Constant Field Values

LOCK_SHARED

protected static final char LOCK_SHARED

See Also:

Constant Field Values

LOCK_SEMI_SHARED

protected static final char LOCK_SEMI_SHARED

See Also:

Constant Field Values

deadlockCnt

protected int deadlockCnt

Number of consecutive deadlocks we have seen

conf

protected HiveConf conf

dbProduct

protected TxnHandler.DatabaseProduct dbProduct

Constructor Detail

TxnHandler

public TxnHandler(HiveConf conf)

Method Detail

getOpenTxnsInfo

public GetOpenTxnsInfoResponse getOpenTxnsInfo()
                                        throws MetaException

Throws:

MetaException

getOpenTxns

public GetOpenTxnsResponse getOpenTxns()
                                throws MetaException

Throws:

MetaException

createValidReadTxnList

public static ValidTxnList createValidReadTxnList(GetOpenTxnsResponse txns,
                                  long currentTxn)

Transform a GetOpenTxnsResponse to a ValidTxnList. This assumes that the caller intends to read the files, and thus treats both open and aborted transactions as invalid.

Parameters:

txns - txn list from the metastore

currentTxn - Current transaction that the user has open. If this is greater than 0 it will be removed from the exceptions list so that the user sees his own transaction as valid.

Returns:

a valid txn list.

openTxns

public OpenTxnsResponse openTxns(OpenTxnRequest rqst)
                          throws MetaException

Throws:

MetaException

abortTxn

public void abortTxn(AbortTxnRequest rqst)
              throws NoSuchTxnException,
                     MetaException

Throws:

NoSuchTxnException

MetaException

commitTxn

public void commitTxn(CommitTxnRequest rqst)
               throws NoSuchTxnException,
                      TxnAbortedException,
                      MetaException

Throws:

NoSuchTxnException

TxnAbortedException

MetaException

lock

public LockResponse lock(LockRequest rqst)
                  throws NoSuchTxnException,
                         TxnAbortedException,
                         MetaException

Throws:

NoSuchTxnException

TxnAbortedException

MetaException

lockNoWait

public LockResponse lockNoWait(LockRequest rqst)
                        throws NoSuchTxnException,
                               TxnAbortedException,
                               MetaException

Throws:

NoSuchTxnException

TxnAbortedException

MetaException

checkLock

public LockResponse checkLock(CheckLockRequest rqst)
                       throws NoSuchTxnException,
                              NoSuchLockException,
                              TxnAbortedException,
                              MetaException

Throws:

NoSuchTxnException

NoSuchLockException

TxnAbortedException

MetaException

unlock

public void unlock(UnlockRequest rqst)
            throws NoSuchLockException,
                   TxnOpenException,
                   MetaException

Throws:

NoSuchLockException

TxnOpenException

MetaException

showLocks

public ShowLocksResponse showLocks(ShowLocksRequest rqst)
                            throws MetaException

Throws:

MetaException

heartbeat

public void heartbeat(HeartbeatRequest ids)
               throws NoSuchTxnException,
                      NoSuchLockException,
                      TxnAbortedException,
                      MetaException

Throws:

NoSuchTxnException

NoSuchLockException

TxnAbortedException

MetaException

heartbeatTxnRange

public HeartbeatTxnRangeResponse heartbeatTxnRange(HeartbeatTxnRangeRequest rqst)
                                            throws MetaException

Throws:

MetaException

compact

public void compact(CompactionRequest rqst)
             throws MetaException

Throws:

MetaException

showCompact

public ShowCompactResponse showCompact(ShowCompactRequest rqst)
                                throws MetaException

Throws:

MetaException

addDynamicPartitions

public void addDynamicPartitions(AddDynamicPartitions rqst)
                          throws NoSuchTxnException,
                                 TxnAbortedException,
                                 MetaException

Throws:

NoSuchTxnException

TxnAbortedException

MetaException

getDbConn

protected Connection getDbConn(int isolationLevel)
                        throws SQLException

Get a connection to the database

Parameters:

isolationLevel - desired isolation level. If you are doing _any_ data modifications you should request serializable, else read committed should be fine.

Returns:

db connection

Throws:

MetaException - if the connection cannot be obtained

SQLException

closeDbConn

protected void closeDbConn(Connection dbConn)

closeStmt

protected void closeStmt(Statement stmt)

Close statement instance.

Parameters:

stmt - statement instance.

checkRetryable

protected void checkRetryable(Connection conn,
                  SQLException e,
                  String caller)
                       throws TxnHandler.RetryException,
                              MetaException

Determine if an exception was such that it makse sense to retry. Unfortunately there is no standard way to do this, so we have to inspect the error messages and catch the telltale signs for each different database.

Parameters:

conn - database connection

e - exception that was thrown.

caller - name of the method calling this

Throws:

TxnHandler.RetryException - when deadlock detected and retry count has not been exceeded. TODO: make "caller" more elaborate like include lockId for example

MetaException

getDbTime

protected long getDbTime(Connection conn)
                  throws MetaException

Determine the current time, using the RDBMS as a source of truth

Parameters:

conn - database connection

Returns:

current time in milliseconds

Throws:

MetaException - if the time cannot be determined

getIdentifierQuoteString

protected String getIdentifierQuoteString(Connection conn)
                                   throws SQLException

Determine the String that should be used to quote identifiers.

Parameters:

conn - Active connection

Returns:

quotes

Throws:

SQLException

determineDatabaseProduct

protected TxnHandler.DatabaseProduct determineDatabaseProduct(Connection conn)
                                                       throws MetaException

Determine the database product type

Parameters:

conn - database connection

Returns:

database product type

Throws:

MetaException - if the type cannot be determined or is unknown