# # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # AMQP 1.0 support for the qpid::messaging API -------------------------------------------- * Connections, Session and Links The protocol used is selected at runtime via the 'protocol' connection property. The recognised values are 'amqp1.0' and 'amqp0-10'. AMQP 0-10 is still the default so in order to use AMQP 1.0 you must explicitly specify this option. The SASL negotiation is optional in AMQP 1.0. If no SASL layer is desired, the sasl_mechanisms connection option can be set to NONE. AMQP 1.0 can be used over SSL, however the messaging client does not at this stage use an AMQP negotiated security layer for that prupose. Peers must expect SSL on the port being used (either exclusively or by being able to detect an SSL header). The container id that the client advertises when establishing the connection can be set through the container_id property on the connection. If not set a UUID will be used. Transactional sessions are not yet supported[1]. The creation of senders or receivers results in the attaching of a link to the peer. The details of the attach, in particular the source and/or target, are controlled through the address string. * Addresses The name specified in the address supplied when creating a sender or receiver is used to set the 'address' field of the target or source respectively. If the subject is specified for a sender it is used the default subject for messages sent without an explicit subject set. If the subject is specified for a receiver it is interpreted as a filter on the set of messages of interest. If it includes a wildcard (i.e. a '*' or a '#') it is sent as a legacy-amqp-topic-binding, if not it is sent as a legacy-amqp-direct-binding. An address of '#' indicates that the sender or receiver being created should be to or from a dynamically created node. I.e. the dynamic flag will be set on the source or target in the attach to request that the broker create a node and return the name of it. The getAddress() method on Sender and Receiver can then be used to set the reply-to on any message (or to establish other links to or from that node). For dynamically created nodes, the dynamic-node-properties can be specified as a nested 'properties' map within the node options. Additionally the durable and type properties in the node map itself are sent if specified. For receivers, where the node is an exchange, the binding can be controlled through the filters for the link. These can be specified through the filter property in the link properties specified in the address. The value of this should be a list of maps, with each map specifying a filter through key-value pairs for name, descriptor (can be specified as numeric or symbolic) and a value. The subject in the address and the selector property within the link properties of the address provide shorter ways of specifying the most common use cases. The source/target capabilities sent by the client can be controlled through a nested list within the node options. Note that capabilities are simply strings (symbols in 1.0 to be precise), not name value pairs. E.g. topic subscriptions can be shared between receivers by requesting the 'shared' capability and ensuring the receivers all specify the same link name. If durable is set in the node properties, then a capability of 'durable' will be requested, meaning the node will not lose messages if its volatile memory is lost. If the type is set, then that will also be passed as a requested capability e.g. 'queue' meaning the node supports queue-like characteristics (stores messages until consumers claim them and allocates messages between competing consumers), 'topic' meaning the node supports classic pub-sub characteristics. * Messages The message-id, correlation-id, user-id, subject, reply-to and content-type fields in the properties section of a 1.0 message can all be set or retreived via accessors of the same name on the Message instance. Likewise for the durable, priority and ttl fields in the header section. An AMQP 1.0 message has a delivery-count field (within the header section) for which there is no direct accessor yet on the Message class. However if the value is greater than 0, then the Message::getRedelivered() method will return true. If Message::setRedelivered() is called with a value of true, then the delivery count will be set to 1, else it will be set to 0. The delivery count can also be retrieved or set through the 'x-amqp-delivery-count' pseudo-property. The application-properties section of a 1.0 message is made available via the properties map of the Message class. Likewise on sending a message the properties map is used to poplulate the application-properties section. There are other fields defined in the AMQP 1.0 message format that as yet do not have direct accessors on the Message class. These are made available on received messages via special keys into the properties map, and can be controlled for outgoing messages by setting the properties with the same keys. The format for the keys in general is x-amqp-. The keys currently in use are: x-amqp-first-acquirer and x-amqp-delivery-count for the header section, and x-amqp-to, x-amqp-absolute-expiry-time, x-amqp-creation-time, x-amqp-group-id, x-amqp-qroup-sequence and x-amqp-reply-to-group-id for the properties section. The delivery- and message- annotations sections can be made available via nested maps with key x-amqp-delivery-annotations and x-amqp-message-annotations respectively by specifying the 'nest_annotations' connection option. Otherwise they will simply be included in the properties. AMQP 1.0 support in qpidd ------------------------- To enable 1.0 support in qpidd, the amqp module must be loaded. This allows the broker to recognise the 1.0 protocol header alongside the 0-10 one. If installed, the module directory should be configured such that this happens automatically. If not, use the --load-module option and point to the amqp.so module. The broker can accept connections with or without and underlying SASL security layer as defined by the 1.0 specification. However if authentication is turned on -- i.e. auth=yes -- then a SASL security layer MUST be used. The broker allows links in both directions to be attached to queues or exchanges. The address in the source or target is resolved by checking whether it matches the name of a queue or an exchange. If there exists a queue and an exchange of the same name, the queue is used but a warning is logged. The incoming and outgoing links attached to the broker can be viewed via the qpid management framework (aka QMF), e.g. with the qpid-config tool: qpid-config list incoming or qpid-config list outgoing If the node is an exchange, then an outgoing link (i.e. messages to travel out from broker) will cause a temporary, link-scoped queue to be created on the broker and bound to the exchange. [See section on 'Topics' below]. The name of the queue will be of the form _. Outgoing links may have a filter set on their source. The filters currently supported by the broker are 'legacy-amqp-direct-binding', 'legacy-amqp-topic-binding', 'legacy-amqp-headers-binding', 'selector-filter' and 'xquery-filter' as defined in the registered extensions. The 'selector-filter' is supported for all nodes. The 'legacy-amqp-direct-binding' and 'legacy-amqp-topic-binding' filters are supported when the node is a topic, direct or xml exchange - where it is used as the binding key when binding the subscription queue to the exchange - and when the node is a queue, where the subscription will then skip over messages that do not match that subject. In the case where 'legacy-amqp-topic-binding' was requested for a direct- or xml- exchange it will be interpreted as a direct binding (and this will be indicated via the filters set on the attach sent by the broker). The 'legacy-amqp-headers-binding' is supported where the node is a headers exchange and implemented as a binding to the exchange. The 'xquery-filter' is supported where the node is an xml exchange and provides the xquery to use in the binding. This can be used in conjunction with a 'legacy-amqp-direct-binding' in order to control the binding ley used for the binding. If the dynamic flag is set on the source or target, then the dynamic-node-properties are inspected to determine the characteristics of the node to be created. The properties that are recognised are the same as accepted via the create qmf method; the 0-10 defined options durable, auto-delete, alternate-exchange, exchange-type and then any qpidd specific options (e.g. qpid.max-count). The broker supports all standard values for the standard lifetime-policy property also. The supported-dist-modes property as defined by the 1.0 specification is used to determine whether a queue or exchange is desired (where the create method uses the 'type' property). If 'move' is specified a queue will be created, if 'copy' is specified an exchange will be created. If this property is not set, then a queue is assumed. Messages sent over AMQP 0-10 will be converted by the broker for sending over AMQP 1.0 and vice versa. The message-id, correlation-id, userid, content-type and content-encoding all map fairly simply in both directions between the properties section in 1.0 and the message-properties in an 0-10 header. Note however that in 0-10 a message-id must be a UUID, and in translating to 0-10 from 1.0 this field will be skipped unless it is a valid UUID. Likewise the priority directly between the field in the header section of a 1.0 message and the corresponding field in the delivery-properties of an 0-10 message. The durable header in a 1.0 message is equivalent to the delivery-mode in the delivery-properties of an 0-10 message, with a value of true in the former being equivalent to a value of 2 in the latter and a value of false in the former equivalent to 1 in the latter. When converting from 0-10 to 1.0, the reply-to will be simply the routing-key if the exchange is not set, or if it is the reply-to address for 1.0 will be composed of the exchange and if specified the routing key (separated by a forward slash). When converting from 1.0 to 0-10, if the address contains a forward slash it is assumed to be of the form exchange/routing key. If not it is assumed to be a simple node name. If that name matches an existing queue, then the resulting 0-10 reply to will have the exchange empty and the routing key populated with the queue name. If not, but the name matches and exchange, then the reply to will have the exchange populated with the node name and the routing key left empty. If the node refers to neither a known queue nor exchange then the resulting reply to will be empty. When converting from 0-10 to 1.0, if the 0-10 message has a non-empty destination, then the subject field in the properties of the 1.0 message is set to the value of the routing-key from the message-properties of the 0-10 message. In the reverse direction, the subject field of the properties section of the 1.0 message is used to populate the routing-key in the message-properties of the 0-10 message. The destination of a 0-10 message is used to populate the 'to' field of the properties section when converting to 1.0, but the reverse translation is not done (as the destination for messages sent out by the broker is controlled by the subscription in 0-10). The application-properties section of a 1.0 message is converted to the application-headers field in the message-properties of an 0-10 message and vice versa. Map and list content is converted between 1.0 and 0-10 formats. Specifically, a 1.0 message with a list- or map- typed AmqpValue section will be converetd into an 0-10 message with the content=type set to amqp/list or amqp/map respectively and the content encoded in the 0-10 typesystem. Likewise an 0-10 message with content-type set to amqp/list or amqp/map will be converted into a 1.0 message with an AmqpValue section containing a list or map respectively, encoded in the 1.0 typesystem. The broker recognises particular capabilities for source and targets. It will only include a given capability in the attach it sends if that has been 'requested' in the attach sent by the peer to trigger the establishment of the link. This gives the peer some means of ensuring their expectations will be met. The 'shared' capability allows subscriptions from an exchange to be shared by multiple receivers. Where this is specified the subscription queue created takes the name of the link (and does not include the container id). The 'durable' capability will be added if the queue or exchange refered to by the source or target is durable. The 'queue' capability will be added if the source or target references a queue. The 'topic' capability will be added if the source or target references an exchange. If the source or target references a queue or a direct exchange the 'legacy-amqp-direct-binding' will be added. If it references a queue or a topic exchange, 'legacy-amqp-topic-binding' will be added. * Topics: a mechanism for controlling subscription queues As there is no standard or obvious mechanism through which to configure subscription queues in AMQP 1.0, a new broker entity of type 'topic' has been added. A topic references an existing exchange and additionally specifies the queue options to use when creating the subscription queue for any receiver link attached to that topic. There can be topics with different names all referencing the same exchange where different policies should be applied to queues. Topics can be created and deleted using the qpid-config tool, e.g. qpid-config add topic my-topic --argument exchange=amq.topic\ --argument qpid.max_count=500 --argument qpid.policy_type=self-destruct If a receiver is established for address 'my-topic/my-key' over 1.0 now, it will result in a subscription queue being created with a limit of 500 messages, that deletes itself (thus ending the subscription) if that limit is exceeded and is bound to 'amq.topic' with the key 'my-key'. * 'Policies': a mechanism for broker configured, on-demand creation of nodes As the core AMQP 1.0 protocol deliberately doesn't cover the creation of nodes, a new mechanism has been added that allows nodes to be created on-demand requiring only broker configuration. A client can establish a link to or from a node, and if the desired target/source matches a preconfigured policy, then the node will be created on demand in accprdance with that policy. There are currently two types of policy: QueuePolicy and TopicPolicy. These allow for the on-demand creation of queues or 'topics' (in the JMS sense of the word), which are currently backed by pre-1.0 AMQP style exchanges. Policies of either type can be created using the qpid-config tool, e.g. qpid-config add QueuePolicy queue_ --argument qpid.max_count=500 will create a policy such that any attempt to establish a link to or from a node that begins with queue_ will result on a queue with the required name being created if it doesn't exist. In this example the queue would have a max depth of 500 messages. * Broker Initiated Links The existing 'federation' mechanism is quite tightly bound to the AMQP 0-10 implementation at present. The AMQP 1.0 support includes a slightly different mechanism that offers the ability to establish links to and from nodes in remote containers, and thus setup the automatic transfer of messages. To begin with, a 'domain' entity must be created describing how to connect to the given external process (i.e. the remote container). This can again be done with qpid-config, e.g. qpid-config add domain another-broker --argument url=host.acme.com Incoming or outgoing links can then be established, e.g. qpid-config add incoming link_a --argument domain=another-broker \ --argument src=some_remote_queue --argument dest=my_local_queue will cause messages on a node named 'some_remote_queue' on the process reachable through the default AMQP port on host.acme.com, to be transferred to a queue, on the qpidd instance these command were issued to, named my_local_queue. Note that at present there is no automatic re-establishment of these broker initiated AMQP 1.0 based links, nor is there any loop prevention mechanism for messages transmitted over them in the event that the links form circular paths. [1] https://issues.apache.org/jira/browse/QPID-4710