This page last changed on Jul 29, 2009 by aidan.

Purpose

This design page describes the low level design for the new interface which is aimed at facilitating encapsulation for the Network code in both the Java Broker & Client.

This is the first step in decoupling the exsiting IO layer from both the surrounding Qpid code and more specifically from the current tie-in to MINA.

This document will provide sufficient information for architecture review and also for input to task breakdown & planning.

Interface Requirements

  1. Provide an API which supports pluggable network layers
  2. Facilitate the replacement of instantiations of MINA classes with an abstraction
  3. Network interface and drivers should be thread model agnostic. The
  4. Ability to set TCP options (see main design doc for details)
  5. Provide support for configuration of related properties including buffer size
  6. The interface will support an SSLEngine

Current design

For details on the current implementation see Current Architecture

New Design

NetworkDriver takes bytes from the network and passes them to the ProtocolEngine. It also accepts bytes from the ProtocolEngine and writes them to the network.

ProtocolEngine accepts bytes from the NetworkDriver and turns them into AMQFrames for processing. It accepts frames and encodes them into bytes which it then hands off to the NetworkDriver.

Design choices

  1. Initial designs will only support TCP (see main design doc for info)
  2. The NetworkDriver.send() method will not block, and neither will the ProtcolEngine.received(). As soon as they have stored the data for later processing they will return.

New network / protocol engine interface in org.apache.qpid.common

In the new version, a NetworkDriver is created by a ProtocolEngine (in the case of outgoing conenctions) or is bound to a socket and creates a ProtocolEngine when new connections are created. The network driver passes raw data to the ProtocolEngine which is responsible for both decoding the frames and processing them. When the ProtocolEngine wishes to send data, it does so by calling the NetworkDriver. The existing mechanisms for frame listeners etc are retained, but are decoupled from the network processing parts.

At the start of a connection the the NetworkDriver will pass data to a ProtocolEngine which will handle protocol negotiation. The implementation will use the existing Sender and Reciever interfaces in org.apache.qpid.transport which will allow the use of the existing alternate transport layer implementations.

The thread of control remains with the network driver up until recieved(), when the ProtocolEngine becomes responsible. The ProtocolEngine should return from recieved after it has accepted the data for processing without blocking the network thread. Conversely, the NetworkDriver should return control to the thread calling send() as soon as possible after accepting the data for writing.

Data comes in from the operating system, is read from the socket by the NetworkDriver and given to the ProtocolEngines received method. The ProtocolEngine is responsible for processing the bytes and interfacing to the rest of the broker or client.

The ProtocolEngine will write bytes to the wire using the NetworkDriver which implements the existing Sender interface from org.apache.qpid.transport

Sender (already exists):

/**
 * This interface is implemented by things which accept data for sending to a remote end point
 */ 
public interface Sender<T>
{

    // Sets the TCP  idle time out
    void setIdleTimeout(long l);

    // Accepts the data for sending
    void send(T msg);

    // Flushes all data pending
    void flush();

    // Closes the connection
    void close();

}

The ProtocolEngine will implement the Reciever interface to be given bytes by the NetworkDriver in it's received method.

Receiever (already exists):

/**
 * This interface is implemented by things which accept data for processing
 */ 
public interface Receiver<T>
{

    // Called when data has been received from the network
    void received(T msg);

    // Called when an exception has occured 
    void exception(Throwable t);

    // Called when the underlying socket has been closed for reading
    void closed();

}

The ProtocolEngine will implement the following interface:

/**
 * A ProtocolEngine is a Receiver for java.nio.ByteBuffers. It takes the data passed to it in the received
 * decodes it and then process the result. 
 */
public interface ProtocolEngine extends Receiver<java.nio.ByteBuffer>
{
   // Sets the network driver providing data for this ProtocolEngine
   void setNetworkDriver (NetworkDriver driver)

   // Returns the remote address of the NetworkDriver
   void SocketAddress getRemoteAddress()

   // Returns number of bytes written
   long getWrittenBytes()

   // Returns number of bytes read
   long getReadBytes()

   // Called by the NetworkDriver when the socket has been closed for reading
   void closed() 

   // Called when the NetworkEngine has not written data for the specified period of time (will trigger a 
   // heartbeat)
   void writerIdle()  
 
   // Called when the NetworkEngine has not read data for the specified period of time (will close the connection)
   void readerIdle()  

   /**
    * Accepts an AMQFrame for writing to the network. The ProtocolEngine encodes the frame into bytes and
    * passes the data onto the NetworkDriver for sending
    */

   void writeFrame(AMQDataBlock frame)
}
public interface ProtocolEngineFactory 
{

  // Returns a new instance of a ProtocolEngine
  ProtocolEngine newProtocolEngine()
  
}

The NetworkDriver will implement the following interface:

public interface NetworkDriver extends Sender<java.nio.ByteBuffer>
{
   // Creates a NetworkDriver which attempts to connect to destination on port and attaches the ProtocolEngine to 
   // it using the SSLEngine if provided
   static NetworkDriver open(int port, InetAddress destination, ProtocolEngine engine, NetworkDriverConfiguration config, SSLEngine engine) throws OpenException;
  
   // listens for incoming connections on the specified ports and address and creates a new NetworkDriver which
   // processes incoming connections with ProtocolEngines created from factory using the SSLEngine if provided
   static void bind (int port, InetAddress[] addresses, ProtocolEngineFactory factory, 
                              NetworkDriverConfiguration config, SSLEngine engine) throws BindException;

   // Returns the remote address of underlying socket
   void SocketAddress getRemoteAddress()

   /**
    * The length of time after which the ProtocolEngines readIdle() method should be called if no data has been 
    * read
    */ 
   void setMaxReadIdle(int idleTime)  
 
   /**
    * The length of time after which the ProtocolEngines writeIdle() method should be called if no data has been 
    * written
    */  
   void setMaxWriteIdle(int idleTime)  

}

The NetworkConfiguration interface provides configuration data for the NetworkDriver:

/**
 * This interface provides a means for NetworkDrivers to configure TCP options such as incoming and outgoing
 * buffer sizes and set particular options on the socket. NetworkDrivers should honour the values returned
 * from here if the underlying implementation supports them.  
 */
public interface NetworkDriverConfiguration 
{ 
   // Taken from Socket 
   boolean getKeepAlive()
   boolean getOOBInline()
   boolean getReuseAddress()
   Integer getSoLinger() // null means off
   int getSoTimeout()
   boolean getTcpNoDelay()
   int getTrafficClass()
 
   // The amount of memory in bytes to allocate to the incoming buffer
   int getReceiveBufferSize(); 
   
   // The amount of memory in bytes to allocate to the outgoing buffer
   int getSendBufferSize(int size); 
} 

Realtionship to existing design

The Current Architecture can be thought of with AMQMinaProtocolSession taking the place of the ProtocolEngine and AMQPFastProtocolHandler being the NetworkDriver. However the seperation of responsibility is not clear between the two and the are both tied directly to MINA.


Document generated by Confluence on May 26, 2010 10:32