This page last changed on May 03, 2009 by ritchiem.

Overview

There are a number of areas of the current Java broker that we would like to look to improve. The primary aims of this work are to simplify the various modules of the broker and clearly identify their interactions according to the identified areas of responsibility.

Where this design refers to the 'current' code base, this is the 0.5 release and the state of trunk at this point. Work completed previously on Flow to Disk is being put to one side for the purpose of this design.

Areas of investigation

We have identified three main areas of to improve.

  1. Message Representation
  2. Storage (Message & Model)
  3. Message Transfer

Message Representation

Summary

Currently the broker uses two implementations of AMQMessageHandle, one transient, one persistent, which actively stores and retrieves the underlying data. The AMQMessage class should provide immutable access to the message data (header and body) stored within the broker, there should be a 1:1 relationship between AMQMessage objects and messages delivered through the broker.

Currently the AMQMessage is responsible for holding a count of references but the responsibility for maintaining this value is spread throughout the code base. This responsibility needs to be given to a single class so that we can more easily reason about and test its functionality.

Approach

Adjusting the existing AMQMessage to ensure that the data it stores is immutable will require moving queue specific data such as Redelivered to the QueueEntry. Converting the existing MessageHandles to a single handle will allow us to easily unload the data as required for Flow to Disk. The additional storage work that was done in the persistent message case will need to move to an object in the Message Transfer layer.
Taking our existing single AMQMessage object we can delegate the responsibility to it for maintaining the reference count. This will prevent any need for a globally locked map and any synchronisation will be reduced to the level of the message.

Storage (Message & Model)

Summary

Currently we have the MessageStore interface which is responsible for all persistent storage for the broker. This not only includes Message Data but also, Queueing details, Queue and Exchange Creations and the bindings between them. This makes for difficulties during broker start up as there are also model configuration located in the configuration files which all needs to be processed before message recovery can be performed.

Approach

The existing MessageStore interface will be slimmed down to be only responsible for storing and retrieving messages. This will then allow a flow to disk mechanism to put the messages on disk independently. A new TransactionLog will be created to record the enqueue and dequeue operations. To store the Queue, Exchange and Binding model information a new ModelConfiguration will be created to persist this data.

MessageStore and TransactionLog

The get a better idea of how these new classes will interact this is the sequence of events that will occur during broker startup. The diagram shows the startup and recovery of the TransactionLog which will retrieve the messages from the MessageStore that are still active. Once the TransactionLog has completed an Orphan detection phase is run on the MessageStore. The exact details of what this process involves can be made a configurable policy decision. However, the initial suggestions are; to leave the messages in the store for tool analysis; the messages can be annotated to record the number of times the message has been detected as an orphan and delete messages that have been marked X number of times; with the introduction of dead letter queues the messages could be moved here.

This sequence highlights how the RecordFactory and TransactionLog would interact. A new Transaction would have Records added or deleted that were created via the RecordFactory. The RecordFactory along with the RecordType are the only two classes that need be Qpid specific. The rest of the Transaction package is a generic log of Records.

Model Configuration

Providing an interface to our current configuration will allow us to de-couple our current code from the various sources that are currently used to store this information.

Message Transfer

Summary

The owner of processing a new message takes to a queue is currently split between a number of objects. IncomingMessage, AMQChannel, and the TransactionalContext. In addition we have the StoreContext which is used to store any required Transactional data.

Approach

The new Transaction object will be created from the context and all operations will be done through this Transaction object. The Transaction will be passed around where the StoreContext currently is and operated on directly.


Document generated by Confluence on May 26, 2010 10:32