/[Apache-SVN]
ViewVC logotype

Revision 787765


Jump to revision: Previous Next
Author: jbellis
Date: Tue Jun 23 17:55:39 2009 UTC (14 years, 10 months ago)
Changed paths: 4
Log Message:
Under heavy load and large column values, we still saw lockups in tcp connection. Here is the problem. The
following code that sets the interest ops seems innocent, but it's the source of the problem. The reason is
that this operation is not atomic. Another thread could sneak in between the reading of the ops and the
setting of it. As a result, some wrong bits could be set.
   key_.interestOps(key_.interestOps() | SelectionKey.OP_READ)

This is a sequence that demonstrates how we can lose the OP_READ bit forever and thus jam the read channel:
1. Thread 1: we want to write a message and in write(Message) we are about to turn on OP_WRITE because the message can't be written in one shot.
2. Thread 2: a read comes in and in read(SelectionKey), we turn off OP_READ and submit the read request to ReadWorkItem in Thread 3.
3. Thread 1: read interestOps and see OP_READ as off.
4. Thread 3: finished processing the read request and turn OP_READ on
5. Thread 1: resumes and turn on OP_WRITE. However, by doing that, we also turned off OP_READ. The read channel is thus blocked forever after this.

patch by Jun Rao; reviewed by jbellis for CASSANDRA-220

Changed paths

Path Details
Directoryincubator/cassandra/branches/cassandra-0.3/src/java/org/apache/cassandra/net/SelectionKeyHandler.java modified , text changed
Directoryincubator/cassandra/branches/cassandra-0.3/src/java/org/apache/cassandra/net/TcpConnection.java modified , text changed
Directoryincubator/cassandra/branches/cassandra-0.3/src/java/org/apache/cassandra/net/UdpConnection.java modified , text changed
Directoryincubator/cassandra/branches/cassandra-0.3/src/java/org/apache/cassandra/net/http/HttpConnection.java modified , text changed

infrastructure at apache.org
ViewVC Help
Powered by ViewVC 1.1.26