Revision 1751977

DERBY-6879: Engine deadlock between XA timeout handling and cleanupOnError

This patch was contributed by Brett Bergquist (brett at thebergquistfamily dot com)

The original problem to be solved is that a connection that is performing a
XA transaction that discovers an error that must be cleaned up is going to
have a lock on a EmbedConnection and will then require a lock on the
XATransactionState.

And if the same XA transaction times out (because of the XA transaction
timer value) while the original XA transaction is being worked, then the
timeout handling is going to have a lock on the XATransactionState and then
require a lock on the EmbedConnection, triggering the deadlock.

The change in the patch fixes this problem by altering the locking on
the XATransactionState when invoked via the timeout by removing the
synchronization on the "cancel" method and instead using inline
synchronization on the "XATransactionState" while its internals are being
altered.

Then, it releases the XATransactionState lock to allow the
EmbedConnection.rollback method to be invoked without the lock.

Finally, it acquires the lock again to finish cleaning up the XATransactionState.