SOAP, REST, and beyond.

I am pleasantly surprised that there indeed seems to be a lot of common ground between the "transport binding matters" people in the REST camp and the "transport binding doesn't and shouldn't  matter" people in the messaging camp. Mark is one of the most vocal proponents of the REST model (and I have a lot of respect for many things he is saying). I have to admit that up until now I considered him a warrior on the REST side of the fence of a "SOAP vs. REST" battle that I never really understood. So, either I didn't understand what he was saying or Mark adjusted his position towards SOAP a bit. Either way would be good ;)

I especially agree with two of his beliefs, given that I understand them right:

I believe that using SOAP as a means for extending underlying application protocols, is the most valuable thing it can be used for.

I believe that using SOAP as a framework from which to build new application protocols has some value, so long as those new protocols are built on transport protocols and not tunneled over other application protocols.

To me, this says essentially: If an application protocol has certain gaps to fill or does explicitly not address specific concerns, wrapping the payload data in SOAP envelopes and utilizing the extensibility of the SOAP envelope (headers) allows to create protocol extensions and protocol rules that help layering additional, required functionality on top of the existing application protocol. If an underlying application protocol already exposes a capability, SOAP-based extensions should not attempt to override them.

This makes very obviously a lot of sense. If you send a message through MQSeries or MSMQ, it would be rather silly to use reliable messaging headers on top of that. If you make those queues transactional, there's little need to enforce an atomic commit protocol between both endpoints in most scenarios, because the queue infrastructure is going to handle that for you. If you simply want to use SOAP to implement RPC-style methods like "createAddress()", "deleteAddress()", "updateAddress()" and "getAddress(arg1 .. argN)", you will find that there's already a set of functionally and semantically equivalent operations in the combination of HTTP and URI addressing and therefore it's not SOAP's job to wrap and obscure this.

The catch is that this only makes very obvious sense for simple scenarios, which are, admittedly, fitting for the vast majority of web services as we see them implemented today. I understand Web services moving forward as a model where the simple scenario of having direct "line of sight" and working in a request/response style will not be sustainable and will have to be replaced with a model that is much more flexible, effectively virtualizes the entire networking infrastructure as we know it today and puts us onto a new level of abstraction to enable interoperability not only vertically inside your applications and horizontally across technology infrastructures, but also across the time-axis.

This means to me: If I build a system today that requires reliable messaging when talking to other systems, it should neither matter whether I am using MSMQ or MQSeries or some other message queuing product, nor should it matter whether I am not using a message queuing product at all, but just approximate reliable messaging on top of HTTP and it should likewise not matter whether I am going to be using something I now can't even envision existing in 4 years from now. To make this work, "capability policies" for the services that I am using and processing hints contained in the message that I am sending need to be able to express such fundamental requirements so that they can be mapped onto the concrete capabilities of an underlying transport and application infrastructure. If the service policy says that reliable messaging is required, this does not necessarily mean that a set of SOAP headers must be used and the protocol must be instantiated on the service as long as the policy requirements are fulfilled. Requirements for security and messaging reliability and data consistency will remain to be important, no matter how much technology changes -- and additional requirements will be added on top of that. Technology will continue to change.

For all of those reasons I am a big fan of the WS-* family of specifications from IBM, Microsoft, "et.al", because all of them provide first and foremost an abstract and extensible framework that addresses something as broad as security or transactions. Only in the next step do these specs provide mappings of today's fashion of implementation technologies that balance between concrete enough to allow proper interop and are loose enough so that vendors have enough headroom to map them into their technology stack. None of these specifications does really worry much about concrete bindings to transports. 

Another reason for the absence of transport bindings in most of these specs is that they are built to work in a virtualized network environment that layers its own routing infrastructure on top of existing networks. True, IP does that today and because it's ubiquitous one would think that virtualization of routing is pure overkill. Still, there are plenty of transport protocols that have a slightly different idea of routing, especially in the realm of so-called "legacy" systems. There is also a huge space that is rapidly growing by the day and for which IP's transport-level model of routing isn't feasible -- because (a) your IP router doesn't know when you are leaving your desktop PC and want to get your IM messages on that cell phone now and (b) you have to switch off your cell phone when you get on an airplane. With a virtualization of routing, the biggest part of the work (getting stuff from here to there) is still done by IP, but the negotiation and setup of application-motivated destinations and routes isn't IP's job. In the presence of application-level routing, applications also get the freedom to choose the best transport for a particular message destination or the next hop on the route. That may be HTTP, but it may also be a Queue, a database store & forward table, a TCP socket, ZMODEM on COM1: or a carrier pigeon.

In the presence of such routing scenarios, where HTTP may just be used as the "Internet-crossing" hop a much longer route, HTTP really is "just a transport" and not an application protocol. Call it abuse, but it's just pragmatic.

All of the above are reasons why I think that in search for a generic interoperability infrastructure, a HTTP centric model like REST is much less desireable than a transport and application protocol (HTTP, SMTP, BEEP, etc.) independent approach -- the same set of arguments is also the reason for why I think that WSDL is a bad idea. WSDL creates a very unfortunate link between transport-bindings, concrete service capability requirements (headers) and message content declarations, which all should be declared in separate places.

Updated: