Steve Loughran Apache Web Services Copyright (c) 2003 Apache Software Foundation This documenet describes a protocol for finding Web Services on a LAN or campus wide network, using multicasting of XML request messages. The underlying payload is similar to that of Service Location Protocol, though the directory agent process is omitted. To scale well, integration with UDDI is required. While it is easy to fault the design, mainly on security and scalability, the initial implementation binds to Axis, automatically exporting running Axis services, enabling local clients to find services without a server. This specification aims to provide dynamic discovery of web services on a Local Area Network. It will not scale up to "the Internet", or indeed a large Intranet, but it lets programs find local implementations of well known services, including UDDI registries. The envisaged uses are embedded web services, workgroup computing systems and to bootstrap server-side clusters in a 'near-zero' configuration environment. Every server exporting services can run a discovery server, responding to queries for the services that it offers. The protocol is not SOAP-centric; you could also use it to enumerate REST objects that the client knows about; The system just maps URIs to URLs and leaves to applications and entities at the end of URLs to work out the details among themselves. The IETF's Service Location Protocol, RFC2608, is an example of how to use multicast discovery in a complex LAN network. Its hierarchy of directory agents are intended to scale, as does the advertisement mechanism, which can keep multicast traffic down. The SLP is gradually acquiring more features, which make implementing it more complex. WS-Discovery could grow to mimic SLP server, but it really expects to hand off complex service location to a UDDI server. Such a server, can of course be located with WS-Discovery. The SLP defines a multicast IP Address 239.255.255.253, that we also use, listening to a different port. The aim is to leverage any administrative routing controls that have been set in place in SLP, that recommends using that address and a TTL of 255; restrictions on packet distance will be made in the network, not in the client. SLPv2 adds scoping, and the option to specify the lifespan of an endpoint being valid, and thus the limit of its cacheability. Again, caching is an effective way to scale, and scoping could be convenient to prevent accidental binding to the wrong implementation. The protocol includes some DHCP integration, the idea being that services should receive scope information from the DHCP service -devices autoscope themselves from the network. of course, if the network does not provide scope, then the service is unscoped. SLP addresses scalability with advertising of directory servers, and a URL based query mechanism to specify searches on the server. In the Web Service world, we may be able to delegate such a task to UDDI servers and the like, although the directory agents effectively create an emergent directory of all located devices, which current UDDI implementations do not. Scoping is essential for scalability. What may be an interesting project would be to provide a UDDI interface to a network of self-advertising web services; letting programs use UDDI interfaces to find services in a dynamic network. Sun's Jini has a LAN scoped discovery mechanism, that is primarily Java based. It could perhaps be extended to locate Web Services, though its bias on Java clients would limit its value. Scalability of Jini is another concern; one it never had to address due to its lack of success outside of some very interesting academic Ubiquitous Computing demonstration projects. Sun's JXTA technology may also be usable for service locations among a peer network. JXTA likes to be configured with the name of a peer system, for the initial bootstrap. JXTA2 has a multiple tier model of peers, that may complicate the model, but make it more workable. HP's e"speak framework had a discovery mechanism for its e-services, a blur of Jini and UDDI. It had a query language; services would register themselves in a vocabulary against which queries could be run. An inconvenience of e"speak was that this was the only way that services could be located. While discovery based location of services is an effective means of locating services, it should not be the only way. Forcing one to go through a registry to find all services is like the Windows font creation API: you can't ask for a font called courier, you have to ask for a fixed-width serif font with a name like courier, and take what you are given, even if it is the wrong font. Universal Plug and Play is a Microsoft-originated specification for device interaction. It goes beyond discovery to mandate the profiles that devices must use, so that they can interact. It is focused at devices, not services, and to make use of the UPnP Intellectual Property, UPnP device vendors must implement at least one standardised device protocol. Thus, even though it adopts SOAP as the interaction mechanism, UPnP is not suitable as a discovery mechanism for arbitrary web services, ignoring technical issues. From a technical standpoint, UPnP is neither good nor bad. It does have a means of advertising functions of a service, but it is not WSDL. The service discovery protocol, SSDP, using a variant of HTTP to make requests, rather than pure XML. The protocol has a default TTL of four, to minimise searches, yet the system lacks the scalability of SLPv2, which keeps traffic down in a complex environment by supporting discovery servers which retain knowledge of currently services, and respond to unicast messages. One feature of UPnP missing from Web Services is notification, yet by its nature notification does not go through firewalls. Using the UPnP notification algorithm would constrain services to only support LAN scoped clients. Although built into Windows XP, UPnP is not broadly popular. One of the weaknesses of the design may be the same as Jini: by redefining how things talk to each other, you need to redesign the entire interoperability stack to have a functional UPnP service. The UPnP device protocols are roughly comparable in complexity to Bluetooth Profiles, yet the fact that everything bar discovery already works today, makes implementing UPnP less than compelling. The Rendezvous framework from Apple is also intended to support device discovery. Rendezvous discounts the notion of implementing a service discovery infrastructure in favour of DNS. In a managed environment, services would dynamically register with the DNS server. In an unmanaged environment, Apple's proposed multicast DNS would let services be self-locating. Apple argue in favour of using DNS for a number of reasons, reasons which are partially compelling. It eliminates the need for new servers on the network, unlike SLP and SLPv2. It also integrates well in a corporate environment, as managed DNS servers take on the workload. This is a marked contrast to UPnP, which is too chatty to use on a busy network. The whole DNS caching and update mechanism also acts as a replicated directory service, which provides high availability. Where Rendezvous falls down is its requirement for a dynamic DNS registry: what does one do at home with static DNS handled by the broadband supplier? One has to install a new DNS server. Without a DNS server, the system will not query beyond the local subnet; their multicast DNS gives all packets a TTL of 1. This limits its use in mildly complex networks, such as one with a wireless access point attached to a home LAN, unless the user is going to go to the effort of setting up a DNS server, at which point the reuse of existing standards becomes a liability, not a feature. Nor is it easy to implement Rendezvous without extending the OS to support multicast DNS. Apple did this for MacOS X, but nobody is seemingly doing it for other platforms. We could (maybe) hack this in as a layer atop Java, implementing a multicast DNS service in Java and using this in the absence of a real DNS system. Similarly, the managed DNS registration and lookup could be implemented in Java with a binding to Axis. A Java implementation of Rendezvous would clearly be an interesting exercise, albeit a potentially complex one. Still, it may be easier than extending the WinNT network stack with the same functionality. A final critique of Rendezvous it that is also weak when it comes to queries: DNS does not really support a complex query syntax. The region-specific hierarchy of DNS provides a single taxonomy for finding services, unlike the multiple-taxonomy approach of UDDI. Where Rendezvous has succeeded is taking multicast service discovery mainstream, by integrating it into the OS and encouraging application developers to work with it. By focusing on discovery, rather than the harder problem of interoperability, it is a simple and manageable specification. Thus it demonstrates the value of a simple discovery mechanism for applications, albeit only in the constrained space of the MacOS family (and perhaps Linux in the near future) Server listens on a well known (potentially IANA-assigned) multicast group, awaiting resolution requests. Client builds a request object, comprising requestID: xsd:int type: xsd:string URI: xsd:anyURL URL: xsd:anyURI scope: xsd:string (can be "") expires: xsd:int (optional) The XML Schema is as follows: <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"> <xs:element name="discovery"> <xs:annotation> <xs:documentation>message of specified type</xs:documentation> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="uri" type="xs:anyURI"> <xs:annotation> <xs:documentation>uri of service</xs:documentation> </xs:annotation> </xs:element> <xs:element name="scope" type="xs:string"> <xs:annotation> <xs:documentation>scope of request, leave empty for 'any' scope</xs:documentation> </xs:annotation> </xs:element> <xs:element name="url" type="xs:anyURI" minOccurs="0"> <xs:annotation> <xs:documentation>endpoint of service</xs:documentation> </xs:annotation> </xs:element> <xs:element name="expires" type="xs:int" minOccurs="0"> <xs:annotation> <xs:documentation>expiry time as time_t integer, always in UTC</xs:documentation> </xs:annotation> </xs:element> <xs:element name="description" type="xs:string" minOccurs="0"> <xs:annotation> <xs:documentation>optional description of endpoint</xs:documentation> </xs:annotation> </xs:element> </xs:sequence> <xs:attribute name="id" type="xs:int" use="required"/> <xs:attribute name="type" type="xs:string" use="required"/> </xs:complexType> </xs:element> </xs:schema> The "type" attribute is the key to the system. The protocol supports a series of verbs. "Find": look up any implementations of the supplied URI. "Found": a found URL mapped from the requested URI. "Advertisement": to be used in any announcement extensions. Requests can be sent out with a TTL of 1,2,3, whatever. If the client wants to find any implementation, rather than enumerate all implementations in a location, then the client should slowly increase the TTL till it gets a response: using the same requestID. Services should ignore repeated requests from the same host with the same request ID. Server receives the "Find" request. If the combination of (senderIP, requestID) is in the list of recent requests, the server does not respond to the request. If the request merits a response, then the server looks to see if it has an entry for that URI in the same scope. If it does it sleeps a short period of time (20-500ms), then posts the response directly back to the client as a unicast datagram. A response is a "Found" message containing the same requestID and URI as the request, adding the URL, and optionally an expiry time and description. The expiry time is a time_t value; UTC seconds since 1970-01-01-0000. The xsd:dateTime value would match were it not for the ongoing cross-platform issues with binding the dateTime to specific time zones. Negative responses, "no-match" responses, do not generate network traffic. The network load does not therefore scale with the number of servers on the network, but with the number of clients using it, and then by the number of successful responses to their requests. There are different ways to extend this; The protocol could support new request types, and have new responses. The content of requests and responses may change too. How can we address this? The proposed solution is XML schemas; if the schema of the request is not recognised, then the server ignores the message. The schema of the response must be bound to that of the request; you cannot change the response schema without also changing the request schema. A future iteration may also add an xsd:any section to the payload What about replacing the current payload with a full SOAP envelope? It would let us add authentication via WS-Security, and do other things. But in the absence of a standard for SOAP-over-multicast-datagrams, and the size limitations of such datagrams, it doesn't seem immediately appealing. One of the interesting challenges is what should recipients of a multicast SOAP message do with mustUnderstand headers they don't understand. The SOAP rules say 'fault' -but that means every recipient is going to send a fault back -this is the wrong thing to do. Where does the system get configured? It can be directly hooked to the SOAP server. WS-Discovery aware endpoints should be able to provide extra configuration information in their deployment metadata, which for Axis implies the deployment descriptor. Web applications should also be able to add or delete entries. This lets a Web Service hide entries, but it also lets a local WS-Discovery server return references to remote services, such as a distant UDDI service. Should WS-Discovery servers be discoverable in their own right, just as SLP directory agents are? Yes, if you want to permit unicast interrogation and management. No if you want to make it harder to find system information and vulnerabilities. A WS-Discovery server may choose to respond to requests to find the WS-Discovery service; this option could be user configurable. The current service string to search for is "service:axis-discovery" The response to such a request is a URL indicating the UDP ports supported; in the absence of a widespread 'udp:' URL schema, we choose to declare the use of this URL schema in our responses. If a URL contains something such as udp://192.168.4.4:1434 then it means that the machine at IP address 192.168.4.4 is responding to unicast datagrams on port 1434. This is an extension of the SLP responses, which introduced tcp: urls, URLS which should not be confused with the .NET remoting URLs, which adopt the same tcp: prefix to mean .NET remoting connections exclusively. This confusion should not have adverse effects -merely serves to demonstrate why meaningful prefixes should be used, and the W3C addressing recommendations (http://www.w3.org/Addressing/) used. The requests and responses must be in the UTF-8 encoding. Multicast IP does scale, but things get very chatty. The protocol needs to minimise chattiness and collisions by having servers not repeat responses, and by adding a random delay before responding. Multicast IP does not work on ad-hoc WLANs; those wireless networks without an access point. Some network stacks refuse to allow applications to bind to a multicast address, though there are hints this may just be a firmware defect. . Could it be used to amplify a DoS attack? Yes, if someone spoofs an IP address and asks for a popular service with a high TTL: all implementations would respond. If servers could determine the # of hops from the client, it could restrict responses to local systems only. As we cannot do that, we can code our client to limit the TTL to a maximum well below 255. Could servers be subjected to a DoS attack? Not really. The computational load of this is minor: an unmarshal of an XML document, a document limited in size to 8192 bytes by the IP protocol itself, a lookup of a hash table, and a generation of an XML response document. A logging attack is possible if the server logs serious failures to the file system; the log system should be configured to not save full details to file, or (better) to use a rolling file system logger. What about security? There is currently no security or authentication in WS-Discovery; after you get an endpoint from it, you have to negotiate with the endpoint to see if you trust it. The risk here is twofold. First, a malicious client could issue may requests, generating server load. Secondly, a malicious server could issue false responses, listing endpoints that were invalid. This could deny clients service, especially if the invalid endpoints were themselves malicious and deliberately slow, acting as a tar-pit endpoint. Security could be addressed with signing of messages and responses, but as well as the size limits of datagrams, signing introduces a need for authentication, which implies some authentication and authorisation framework. The obvious solutions would be XML-Signature and perhaps even WS-Security, the latter needing a SOAP-esque payload first.