AOP demo.

MS Finland put up a (long, 30min.) Windows Media 9 series stream of the demo that I did in my AOP talk over in Helsinki. And Matt Powell from MSDN is so nice to mention the things I demo there in his MSDN TV episode on SoapExtensionReflectors/Importers. My demo code is here.

Reconsidering UDDI

For the record: I was wrong when I said this here. When I started to look into and think about service oriented architectures*, I understood the relevance of UDDI.

To cite Don (more than half a year ago): "If WSDL and UDDI were both hanging from a cliff and I could only save one of them, it would be UDDI."  And in case someone didn't notice, yet, I really, really want WSDL to go very near that cliff.

[*this is the most current deck for the SOA talk I do on the Microsoft EMEA Architect's Tour 2003. Check out slide 20 for an illustration of how WS-Routing, WS-Addressing, WS-Policy, message contracts and UDDI fit into a single picture]

Messaging rules.

I haven't blogged or even read blogs for almost two weeks. Too much time on the road. Reading (UK), Zürich, Hannover, Bled (Slovenia), Amsterdam, Budapest. Next week it's Bad Ems and then it'll be Athens, Madrid and Milano in the following two weeks.

Anyways... I find it very interesting that more and more people are seeing the light in regards to the true potential of Web Services (could someone please drop the "Web" term from that?): Messaging. We're getting there ...

IIS 6.0 process model in a nutshell

Sam Gentile has blogged a compact overview on the IIS 6.0 process model. Because pictures say more than a thousand words, I recommend that you get this PPT file from Bill Staples (Group Program Manager for IIS at MS) in addition to reading what Sam has to say.

The bigger language problem 

C#? C++? VB? Java? English! Whenever I speak at conferences across Europe (right now that's about twice a week), every other attendee's first comment when talking to me after any speech will not be a technical one. Instead they say "your English is fantastic". People seem indeed surprised s'at mei englesh iss not vot s'ey vutt expeckt.

I don't think that it is a "fantastic skill", but rather just part of doing my job right. Having reasonable control of the language that is the "one and only" of the IT business -- and that happens to be (American) English -- is essential. I was lucky enough that my former employer sent me on a two year assignment to New York City in '95/'96 and that was the time where I got a sobering reality check on my "school English", picked up all the idioms that let me understand what the Americans are saying between the lines, and (somewhat unfortunately) had all my English teachers' efforts to provide me with a polished "British" spoiled for good.

There is a huge problem here. There are unknown (but large) numbers of very bright people out there, who simply cannot make themselves heard to a broader, world-wide audience just because they don't write and speak English "well enough". Also, English language skills are essential to understand anything happening on the "cutting edge" of computing, because there is always a considerable lag for specifications, books and white papers to appear in any other language than English.

Right now, it's more important for my company that people know English than knowing specifically Java, C#, VB, C++ or SmallTalk. If they have a good understanding of programming as such, it's a tiny step to switch between languages. However, even if they are fabulously eloquent and well read in German, they still may not be able to get a single understandable English sentence together and, just as bad, may miss implied statements in English material they read.

Can we fix this? The answer is certainly not to lament about "language imperialism". The answer is to deal with the facts as they present themselves today and for the future as far as we can see ahead. Being "proudly European", I think that in order to be able to successfully compete with America in this industry, the Europeans, especially the French and the Germans, will have to start to understand that technical degrees must come with proper education in English on a level that's far beyond current "school English" and that a good deal of technical material should indeed be taught and tested in English. Likewise, students need to understand that it's not enough to be a German or French tech-geek. "My English is not very good" is unfortunately not a valid excuse if you are in the IT industry -- it's indeed even worse than saying "my C++ is not very good" when you apply for a systems programmer job.

What's really bad is that I am writing this in English. Sorry, I can't write it in all the languages in which it would matter to say this. Therefore I have to opt for a somewhat neutral choice.....

Sam has a new essay on nouns vs. verbs. Thank you, Sam. All roads lead to Rome. Rome doesn't care. ;)

W3C Web Services Description Group makes a good move.

While they are yet not as far as to they drop the ball, go home and call WSDL dead ;) but there's a noteworthy action item on the activity's task list as per their most recent teleconf call: "2003-03-13: Don [presumably Don Mullen from TIBCO and not Don Box from Microsoft] will write a proposal for annotating schema with part information.", which is an action item apparently added when they were talking about killing the <wsdl:message/> element. I am all for that.

When we get to the point that WSDL's job becomes not to describe a "Web Service", but rather only the applicable message exchange patterns (MEP) for a conversation between two points (which both may engage in other conversations following different MEP's), would kill the <wsdl:service/> binding (see WS-Addressing), and would then be appropriately renamed, I will be fairly happy with it.

Yet, looking at it that way, there should probably be only one WG for this instead of two. There's a Web services description and a Web services choreography WG and that's somewhat like one language designer designing "method" and another guy designing "interface" and looking at it as if they were only loosely related. Of course I may be entirely off base...

newtelligence should probably really join W3C and WS-I, so that I am not just a loud, complaining troublemaker, but an active, constructive troublemaker.

First look at WS-Addressing

WS-Addressing, which gives us a lightweight mechanism to communicate references to endpoints, is, in my view and contrary to what the spec itself explicitly says, yet another nail in the coffin of WSDL. That's because it essentially overrides the <wsdl:service/> element for any scenario, in which the message contract shall be applicable to several different or changing endpoints or intermediaries (in other words: any endpoint that wants to support transport-independent redirection/routing)

WS-Addressing is also a good choice for configuring proxies. You typically develop using a "development cloud" of a web service and not using the actual production endpoint references. If you use WS-Addressing endpoint references as part of your configuration infrastructure, you can flip back and forth between a development and production clouds without recompiles and do so in a standard way now. Until this morning, we had no standard way of expressing this; now we do and that means that framework code for proxies can be written that allows dynamic switching. The roadmap document promises a WS-EndpointResolution spec to further formalize this.

Completing the picture

So, here we have it. A web service UDDI registration (or LDAP entry) should really have three technical entries: A message contract description (WSDL for the time being), a policy description (WS-Policy) and a qualified set of endpoint references (WS-Addressing). The policy refers to the capabilities of the service (WS-Transactions, WS-Security, WS-ReliableMessaging, ...). As per the roadmap, I expect this composition to be expressed using the upcoming WS-MetadataExchange.

For a routed infrastructure on top of registries, you can have then local registries with local WS-Addressing endpoint references that identify endpoints which themselves will use "regional" or "global" registries to find further hops. For a routed infrastructure with dynamic reconfiguration, you can send something like WS-Referral-encapsulated WS-Addressing references around that will update routers. I expect that we will see adjustments to WS-Referral and WS-Routing to eliminate current overlaps with WS-Addressing. The message contract will never have to change, the policies are according to the requirements of specific endpoints and the endpoint defintions you use may vary depending on who, where and how connected you are. 

Today should be a very happy day in Web services land.

WS-ReliableMessaging and WS-Addressing are public!

WS-ReliableMessaging: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnglobspec/html/wsrmspecindex.asp?frame=true
WS-Addressing: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnglobspec/html/wsaddressingspecindex.asp?frame=true

Here's BEA's, TIBCO's, IBM's and Microsoft's take on how to do reliable messaging on web services. There are more than subtle differences compared to the "WS-Reliability" spec by NEC, Sun, Oracle, etc: WS-ReliableMessaging is extensible, takes WS-Security and WS-Policy into account and it does not have (less is sometime more) any specific transport bindings -- and it's routable (read on).

I will likely write about a couple of observations later, but here's a little "reading guide" for a single sentence in section 3.1 saying "There MUST be no more than one <Sequence> element in any message". That language is a bit confusing an could be worded a bit better, because what it really says, when read having SOAP 1.2's "role" or SOAP 1.1's "actor" model present in your mind is indeed "There MUST be no more than one <Sequence> element targeted at one role in any message".

That's because in the presence of roles, all headers become technically "invisible" to any but the targeted role and there is indeed never more than one <Sequence> element existing in the message as far as a certain role is concerned. It'd be helpful if the language was a bit clearer there, but SOAP actually takes care of enabling roles for this, even in the presence of that sentence.

Why are roles relevant? Because this now enables reliable delivery in the presence of routing intermediaries, which want to establish reliable links between them. If you have A->B->C->D with reliable messaging being used for the end-to-end connection A->D and the hop B->C shall be individually outfitted with reliable messaging for purposes of optimization, you can do that by specifying roles on an additional <Sequence> that applies to B and C.

InfoPath Kool-Aid. I rushed home today to sit down and enjoy a big glass of InfoPath Kool-Aid. I can certainly see that there is much goodness, but I'm still trying to get my head around the implications for my daily life. So far the only blog-worthy observation I have is that the .XSN files it produces aren't XML. I was expecting to see angle brackets. I hoped to design a form that manipulated InfoPath metadata. Curious. [Brian Graf's Weblog]

Just open the *.xsn file with WinZip. It's a CAB file ;)

AOP Serviced Components status update. I am still working on it. I've managed to get around most of the big issues and I am now working on stability issues, most prominently on a pretty huge memory leak that's caused by some side effect of what I am doing down there, in the dark dungeons between RCWs, CCWs and more CCWs. While I do it, I keep showing it at presentations around Europe and get quite positive feedback.

SOAP, REST, and beyond.

I am pleasantly surprised that there indeed seems to be a lot of common ground between the "transport binding matters" people in the REST camp and the "transport binding doesn't and shouldn't  matter" people in the messaging camp. Mark is one of the most vocal proponents of the REST model (and I have a lot of respect for many things he is saying). I have to admit that up until now I considered him a warrior on the REST side of the fence of a "SOAP vs. REST" battle that I never really understood. So, either I didn't understand what he was saying or Mark adjusted his position towards SOAP a bit. Either way would be good ;)

I especially agree with two of his beliefs, given that I understand them right:

I believe that using SOAP as a means for extending underlying application protocols, is the most valuable thing it can be used for.

I believe that using SOAP as a framework from which to build new application protocols has some value, so long as those new protocols are built on transport protocols and not tunneled over other application protocols.

To me, this says essentially: If an application protocol has certain gaps to fill or does explicitly not address specific concerns, wrapping the payload data in SOAP envelopes and utilizing the extensibility of the SOAP envelope (headers) allows to create protocol extensions and protocol rules that help layering additional, required functionality on top of the existing application protocol. If an underlying application protocol already exposes a capability, SOAP-based extensions should not attempt to override them.

This makes very obviously a lot of sense. If you send a message through MQSeries or MSMQ, it would be rather silly to use reliable messaging headers on top of that. If you make those queues transactional, there's little need to enforce an atomic commit protocol between both endpoints in most scenarios, because the queue infrastructure is going to handle that for you. If you simply want to use SOAP to implement RPC-style methods like "createAddress()", "deleteAddress()", "updateAddress()" and "getAddress(arg1 .. argN)", you will find that there's already a set of functionally and semantically equivalent operations in the combination of HTTP and URI addressing and therefore it's not SOAP's job to wrap and obscure this.

The catch is that this only makes very obvious sense for simple scenarios, which are, admittedly, fitting for the vast majority of web services as we see them implemented today. I understand Web services moving forward as a model where the simple scenario of having direct "line of sight" and working in a request/response style will not be sustainable and will have to be replaced with a model that is much more flexible, effectively virtualizes the entire networking infrastructure as we know it today and puts us onto a new level of abstraction to enable interoperability not only vertically inside your applications and horizontally across technology infrastructures, but also across the time-axis.

This means to me: If I build a system today that requires reliable messaging when talking to other systems, it should neither matter whether I am using MSMQ or MQSeries or some other message queuing product, nor should it matter whether I am not using a message queuing product at all, but just approximate reliable messaging on top of HTTP and it should likewise not matter whether I am going to be using something I now can't even envision existing in 4 years from now. To make this work, "capability policies" for the services that I am using and processing hints contained in the message that I am sending need to be able to express such fundamental requirements so that they can be mapped onto the concrete capabilities of an underlying transport and application infrastructure. If the service policy says that reliable messaging is required, this does not necessarily mean that a set of SOAP headers must be used and the protocol must be instantiated on the service as long as the policy requirements are fulfilled. Requirements for security and messaging reliability and data consistency will remain to be important, no matter how much technology changes -- and additional requirements will be added on top of that. Technology will continue to change.

For all of those reasons I am a big fan of the WS-* family of specifications from IBM, Microsoft, "et.al", because all of them provide first and foremost an abstract and extensible framework that addresses something as broad as security or transactions. Only in the next step do these specs provide mappings of today's fashion of implementation technologies that balance between concrete enough to allow proper interop and are loose enough so that vendors have enough headroom to map them into their technology stack. None of these specifications does really worry much about concrete bindings to transports. 

Another reason for the absence of transport bindings in most of these specs is that they are built to work in a virtualized network environment that layers its own routing infrastructure on top of existing networks. True, IP does that today and because it's ubiquitous one would think that virtualization of routing is pure overkill. Still, there are plenty of transport protocols that have a slightly different idea of routing, especially in the realm of so-called "legacy" systems. There is also a huge space that is rapidly growing by the day and for which IP's transport-level model of routing isn't feasible -- because (a) your IP router doesn't know when you are leaving your desktop PC and want to get your IM messages on that cell phone now and (b) you have to switch off your cell phone when you get on an airplane. With a virtualization of routing, the biggest part of the work (getting stuff from here to there) is still done by IP, but the negotiation and setup of application-motivated destinations and routes isn't IP's job. In the presence of application-level routing, applications also get the freedom to choose the best transport for a particular message destination or the next hop on the route. That may be HTTP, but it may also be a Queue, a database store & forward table, a TCP socket, ZMODEM on COM1: or a carrier pigeon.

In the presence of such routing scenarios, where HTTP may just be used as the "Internet-crossing" hop a much longer route, HTTP really is "just a transport" and not an application protocol. Call it abuse, but it's just pragmatic.

All of the above are reasons why I think that in search for a generic interoperability infrastructure, a HTTP centric model like REST is much less desireable than a transport and application protocol (HTTP, SMTP, BEEP, etc.) independent approach -- the same set of arguments is also the reason for why I think that WSDL is a bad idea. WSDL creates a very unfortunate link between transport-bindings, concrete service capability requirements (headers) and message content declarations, which all should be declared in separate places.

Just how RESTful is TV?

I've spent some time this morning going though a bunch of web pages explaining and debating RESTful web services. The claim that I see repetitively is that REST is scalable, because it works like the web works and that has proven to be massively scalable. What I don't get together is the basic idea that "every thing is a uniquely identifiable resource" with the bare fact that the seemingly limitless scalability of the largest websites is indeed an illusion created by folks like those at Akamai, which take replicated content around the globe and bring it as close as possible to the clients. It's a lot like TV. TV scales endlessly as long as you are in reach of a broadcast antenna or a satellite beam. It all works beautifully in one direction. However, we don't really have fully interactive TV, yet, because the whole backchannel story is a major problem. It's a pure scalability problem.

If you GET something from the web, you think you are getting it from a uniquely identifiable bucket, but you are indeed quite often smartly spoofed elsewhere and you will get nearly current information. If you PUT or POST something, however, the data will ultimately have to end up in a single bucket -- and unfortunately there is no magically better concurrency story in REST just because it sits directly 'in' the web technology stack.

I will have to admit that I had a weak moment yesterday where I thought that the whole REST story makes a lot of sense. I was thinking of ways to really make it dead simple to use a tool I am currently playing with in the context of a larger application and REST has a lot of appeal there. The "create" story of REST using HTTP PUT is very pretty for this and HTTP GET speaks for itself. It all falls apart when you think about concurrent updates and concurrent deletes. These simply can't exist in such a world. As long as you only create stuff and reference stuff, all is fine.

One of the core problems with turning the REST model into an universal approach for dealing with data is that it is, in all its simplicity and purity, absolutely unaware of what data it is dealing with and it has no sense of the sort "identity" that grows from within the data itself. Once you assign a URL like 'http://www.tempuri.org/store/items/castings/slagpots/7829' to a data set in reply to a create request (PUT), you may essentially be locking down a whole set of properties of this data set that can never be changed if the URL shall not lose all of its own organizational qualities. Even more so, the data gets totally locked down once you follow this path of "object identity" to the last consequence and allow direct references into the data like through this URI 'http://www.tempuri.org/store/items/castings/slagpots/7829!item/manufacturer/foundrylocation/@country' where the second portion (here separated by a '!') is an XPath reference into an assumed XML InfoSet located at aforementioned URL. If two parties working on a data snapshot (that's all you GET) were competing for updates here, and the first party to hit the resource would do a wholesale replacement of the 'manufacturer' data, the second party would now only manipulate a well-known place (/item/manufacturer/foundrylocation/@country), but a wholly different data context. Taken to the last consequence of every data facet being uniquely identifiable through a URI (which it then should be and is) this model can't properly deal with any modification concurrency. [Note that we have a 'manufacturer of this item' relationship here where the 'country' is immediately relevant to this exact type of item and not to the manufacturer in general - so we can't factor the manufacturer data out to a different URL]

As said, REST can deal beautifully with creating and referencing data sets, but how do you pass references around for which any guarantee of currency can given? In the end, I am interested in a data record that represents a specific data item of interest. In the absence of updatability, I will still always know that there is a URI that points to the most current data set for this data item, but unless I have created this item myself, I can never know which URI that is. So, what are the limits of data granularity to which REST applies? How do you define boundaries of data sets in a way that concurrency could work in some way and how do you define the relationship between the data and the URI that identifies it?

The theory behind REST sounds a lot like the theory behind distributed objects (not distributed systems), where you have one virtual thing that represents one real thing. An 'item' in a catalog is a uniquely identifiable thing with its own object identity. I used to be a big fan of this idea until I found it not to be working for a lot of aspects, including, most prominently, high scalability alongside proper concurrency management. This basic idea is very much in the spirit of what CORBA is all about -- and, don't get me wrong here, that model is very good and very valid for a big number of use cases, especially in real-time or near real-time enviornments that deal with a lot of transient data. [As a sidenote, it's quite interesting that if you go and look in Google's Usenet archives and look for the term CORBA combined with the names of some of the most prominent REST proponents you will find what you would expect.]

I have written about the REST'ish view of HTTP being the one and only protocol in the context of transactions and reliability quite a while ago and that perception hasn't changed. Werner Vogels had some additional comments on the reliable messaging angle two weeks ago.

All that being said - I do believe there is a place for REST and plenty of use-cases. These use-cases are likely very much those for which CORBA's fundamental idea of objects already was a very good model. And there are plenty of use-cases for stateless and stateful RPC (in the absence of the 'single thing' idea) and there are plenty for messaging. There is a place for SOAP, which is at its core simply an envelope for annotating data as it crosses system and organization boundaries, in all of these models. That's what Juliet's night is really about.

InfoPath Fodder from the Boneyard

I am sitting in my home office and I am (after being told so by the emperor of the house) cleaning out old stuff, sorting papers, etc. pp.   This includes checking, invalidating and throwing out a couple hundred of old CDs and consolidating all that's worth keeping. Aaaaaaanyways.... I just did something very, very scary and outrageous: I installed my PDC 2001 copy of the Hailstorm SDK to get some schema fodder for InfoPath. Uh! Oh! Where does that go?