I am in Budapest today and I am just done with my Indigo talk (you can find the slides at http://codezone.info under “Talks”), having done it for the 6th time on this tour throughout Europe. After the events Den Haag, Oslo, Copenhagen, Helsinki and Geneva, I still find Indigo a very difficult topic to talk about on this tour. It’s not about technology or because my talk doesn’t work: It’s about whether people think it’s relevant to their work.

The true challenge is to explain to the developers we meet that Indigo is going to be very important for them down the road. I find that when I talk to developers on this tour or look at their evaluation forms that very many of them apparently still write fairly compact (to avoid the word monolithic) ASP.NET applications or Windows Forms applications that use a conservative client/server approach. All presentation and logic resides in one tier and the only remote component worth mentioning is the database. That means that the majority of the folks sitting in my talks hasn’t even touched one of the existing distributed technology stacks that Indigo is set to replace.

The difficulty presenting Indigo on this tour – alongside sexy stuff like declarative UI programming with spinning Windows and Videos with alpha-blending in Avalon and googlefast cross-media searches across all of your local storage media as in WinFS – is that Indigo is about things that are hidden inside applications and do not surface to the user. Stuff that drives server-applications is sometimes hard to understand without knowing the architectural background and the motivations. (Sidenote: A while ago I heard a rumor from a usually trustworthy source that the spinning balls in the COM+ Explorer exist because COM+ was horribly hard to demo as well and the spinning balls provided a good way of visualizing that stuff was happening.)

The ideal talk for an unsuspecting audience with little knowledge in distributed systems would have to sell the whole idea of distributed systems to boot, the experiences and errors made, the reasons for why Web services are a good thing, the problems creating the motivation for and the principles of service oriented architectures, a set of some tangible application examples and use cases along with the solutions that Indigo provides; all of that in the same talk and within 75 minutes. And that in a way that developers get to see code and demos, too. That sort of talk would span about 20 years of distributed computing history. I am not sure this fits in 75 minutes. Therefore I think I will have to be happy with only a fraction of the audience being interested and/or willing to appreciate the things that I am talking about here. 

Very many folks think that the topics I am talking about are only relevant to “big apps” and have a hard time seeing the benefits of something like Indigo – much in the same way as it is with Enterprise Services or Web Services.

If you believe Don Box, who said at PDC that Indigo will ship at some point between Whidbey and Longhorn, and think about the implications of that, Indigo is in fact relevant to everyone writing applications that expose functionality to other applications in some way – now or at least quite soon. The first ship vehicle for Indigo will be, if Don’s statement holds water in its consequences, some service pack or upgrade pack for Windows Server 2003 and Windows XP. That means nothing less than the entire application infrastructure of Windows Server 2003 is getting a major upgrade probably in a year or so from now.

If you are writing applications using ASMX, Remoting or Enterprise Services today, the impact of Indigo’s arrival can be immediate if you want to make it so. If you code your applications cleverly today (following guidelines explained by Joe Long here or in my talk) and don’t play too many tricks on the infrastructure – for instance by using the Remoting extensibility points – you should have a fairly smooth upgrade path to Indigo. The goal is that upgrading code will be simple and mechanical in most cases.

Categories: EMEA Longhorn Preview | Indigo

January 24, 2004
@ 09:45 PM

Don says that BEA's Deputy CTO has missed the cluetrain. I absolutely agree with Don's opinion on this article and what's even worse than the things said is what the article implies. If that is BEA's official position, this is nothing less than an outing that they are passengers in the backseat of a car that is driven by IBM and Microsoft (switching drivers every once in a while) and that they're neither behind the spirit of the whole undertaking nor do they fully understand the specifications they have put their names on. Integration or standardization on the API level has failed miserably in countless attempts and any middleware company (including BEA) that is out there to compete on features must go beyond the least common denominator approach to win over customers. Does BEA have Indigo envy?

Categories: Technology | Indigo

January 24, 2004
@ 08:08 PM

I admit: I haven't read anybody's blog in over 1 1/2 months. I haven't posted much. I actually got tired of blogging and reading blogs for a while. Multiple projects under pressure don't go well with blogging for me. I don't even have an RSS reader on this new machine yet (I am going to change that within the hour). There's a lot of stuff to write about and I plan to write lots about the FABRIQ (which is coming along nicely), Transactions and Indigo while on the road. I've got my blogging mojo back: Yeah, baby.    

Categories: Blog | Other Stuff

The evolution of in-memory concept of messages in the managed Microsoft Web Services stack(s) is quite interesting to look at. When you compare the concepts of System.Web.Services (ASMX), Microsoft.Web.Services (WSE) and System.MessageBus (Indigo M4), you'll find that this most fundamental element has undergone some interesting changes and that the Indigo M4 incarnation of "Message" is actually a bit surprising in its design.

ASMX

In the core ASP.NET Web Services model (nicknamed ASMX), the concept of an in-memory message doesn't really surface anywhere in the programming model unless you use the ASMX extensibility mechanism. The abstract SoapMessage class, which comes in concrete SoapClientMessage and SoapServerMessage flavors has two fundamental states that depend on the message stage that the message is inspected in: The message is either unparsed or parsed (some say "cracked").

If it's parsed you can get at the parameters that are being passed to the server or are about to be returned to the client, but the original XML data stream of the message is no longer available and all headers have likewise either been mapped onto objects or lumped into a "unknown headers" array. if the message is unparsed, all you get is an text stream that you'll have to parse yourself. If you want to add, remove or modify headers while processing a message in an extension, you will have to read and parse your copy of the input stream (the message text) and write the resulting mesage to an output stream that's handed onwards to the next extension or to the infrastructure. In essence that means that if you had two or three ASMX-style SOAP extensions that implement security, addressing and routing functionality, you'd be parsing the message three times and serializing it three times just so that the infrastructure would parse it yet again. Not so good.

WSE

The Web Services Enhancements (WSE) have a simple, but very effective fix for that problem. The WSE team needed to use the ASMX extensibility point but found that if they'd build all their required extensions using the ASMX model, they'd run into that obvious performance problem. Therefore, WSE has its own pipeline and its own extensibility mechanism that plugs as one big extension into ASMX and when you write extensions (handlers) for WSE, you don't get a stream but an in-memory info-set in form of a SoapEnvelope (that is derived from System.Xml.XmlDocument and therefore a DOM). Parsing the XML text just once and have all processing steps work on a shared in-memory object-model seems optimal. Can it really get any better than "parse once" as WSE does it?

Indigo

When you look at the Indigo concept of Message (the Message class in the next milestone will be the same in spirit, similar in concept and different in detail and simpler as a result), you'll find that it doesn't contain a reference to an XmlDocument or some other DOM-like structure. The Indigo message contains a collection of headers (which in the M4 milestone also come in an "in-memory only" flavor) and a content object, which has, as its most important member, an XmlReader-typed Reader property.

When I learned about this design decision a while ago, I was a bit puzzled why that's so. It appeared clear to me that if you kept the message parsed in a DOM, you'd have a good solution if you want to hand the message down a chain of extensibility points, because you don't need to reparse. The magic sentence that woke me up was "We need to support streaming". And then it clicked.

Assume you want to receive a 1GB video stream over an Indigo TCP multicast or UDP connection (even if you think that's a silly idea - work with me here). Because Indigo will represent the message containing that video as an XML Infoset (mind that this doesn't imply that we're talking about base64-encoded content in an UTF-8 angle bracket document and therefore 2GB on the wire), we've got some problems if there was a DOM based solution. A DOM like XmlDocument is only ready for business when it has seen the end tag of its source stream. This is not so good for streams of that size, because you surely would want to see the video stream as it downloads and, if the video stream is a live broadcast, there may simply be no defined end: The message may have a virtually infinite size with the "end-tag" being expected just shortly before judgment day.

There's something philosophically interesting about a message relaying a 24*7*365 video stream where the binary content inside the message body starts with the current video broadcast bits as of the time the message is generated and then never ends. The message can indeed be treated as being well-formed XML because there is always a theoretical end to it. The end-tag just happens to be a couple of "bit-years" away.

Back to the message design: When Indigo gets its hands on a transport stream it layers a Message object over the raw bits available on the message using an XmlReader. Then it peeks into the message and parses soap:Envelope and everything inside soap:Header. The headers it finds go into the in-memory header collection. Once it sees soap:Body, Indigo stops and backs off. The result of this is a partially parsed in-memory message for which all headers are available in memory and the body of the message is left sitting in an XmlReader. When the XmlReader sits on top of a NetworkStream, we now have a construct where Indigo can already work on the message and its control information (headers) while the network socket is still open and the rest of the message is still arriving (or portions haven't even been sent by the other party).

Unless an infrastructure extension must touch the body (in-message body encryption or signature do indeed spoil the party here), Indigo can process the message, just ignore the body portion and hand it to the application endpoint for processing as-is. When the application endpoint reads the message through the XmlReader it therefore pulls the bits directly off the wire. Another variant of this, and the case where it really gets interesting, is that using this technique, arbitrary large data streams can be routed over multiple Indigo hops using virtualized WS-Addressing addressing where every intermediary server just forwards the bits to the next hop as they arrive. Combine this with publish and subscribe services and Indigo's broadcasting abilities and this is getting really sexy for all sorts of applications that need to traverse transport-level obstacles such as firewalls or where you simply can't use IP.     

For business applications, this support for very large messages is not only very interesting but actually vital for a lot of applications. In our BizTalk workshops we've had quite a few customers who exchange catalogs for engineering parts with other parties. These catalogs easily exceed 1GB in size on the wire. If you want to expand those messages up into a DOM you've got a problem. Consequently, neither WSE nor ASMX nor BizTalk Server nor any other DOM based solution that isn't running on a well equipped 64-bit box can successfully handle such real-customer-scenario messages. Once messages support streaming, you have that sort of flexibility.

The problem that remains with XmlReader is that once you touch the body, things get a bit more complex than with a DOM representation. The XmlReader is a "read once" construct that usually can't be reset to its initial state. That is specifically true if the reader sits on top of a network stream and returns the translated bits as they arrive. Once you touch the message content is the infrastructure, the message is therefore "consumed" and can't be used for further processing. The good news is, though, that if you buffer the message content into a DOM, you can layer an XmlNodeReader over the DOM's document element and forward the message with that reader. If you only need to read parts of the message or if you don't want to use the DOM, you can layer a custom XML reader over a combination of your buffer data and the original XmlReader.

Categories: Technology | Indigo | Web Services

January 24, 2004
@ 09:54 AM

The Microsoft Developer Days 2004 in Den Haag (The Hague) were a great event. Not so much fun was going there (the train from Utrecht was split in two trains on the way and I ended up in Rotterdam instead of Den Haag at first) and getting back (the train from Venlo to Düsseldorf simply didn't go because of "technical difficulties" so I had to take a rather expensive cab home). 

I've had lots of interesting discussions and the result of one was that I might be speaking at the SDGN's CttM conference. I'll definitely be back for the second run of the Architect's Forum in Zeewolde in March 29th.  

De SDGN heft gezegt dat ik nu moet genoeg Nederlands lere omdat ik mij CttM presentatie in de Nederlandse taal kan doen, maar ik weet niet of ze bereid zijn om mij zovel tijd voor een presentatie te geve zo dat ik ook lang genoeg voor iede enkele woord kan zoeke. :)     

My talk on Indigo apparently went well for the audience and one of my fellow RDs even said that he learned more about Indigo in my talk than at the PDC (that's because I consolidated the PDC slides and therefore have it "all at once"), but personally I was a bit unhappy with it. Didn't flow right. Two slides too much, one slide missing (I need to explain "Dialogs"). This will be fixed for the next stop in Oslo on Monday.

Categories: Talks | EMEA Longhorn Preview | Indigo

I leaving shortly for Den Haag for the first installment of the Longhorn Developer Preview Tour throughout Europe as part of the Dutch Developer Days 2004. We start tomorrow and I am quite excited since this is the first time I will speak about Indigo in any detail to a larger audience. I've witnessed Indigo "forming" from a distance when the team was still in "stealth mode" and it's great to see how it comes along.

But be forewarned: In my talk there will be no live demos. I have 75 minutes for the talk and I had to decide whether I concentrate on explaining the "M5" milestone that is currently in development in Redmond and which implements the (likely) final programming model or whether I allocate more time to the M4 model found in the PDC build. The decision that I made was that M4 is so different from M5 that unless you want to get a major degree in Longhorn development history or have way too much time on your hands, learning and therefore showing M4 code is almost pointless. I will show code, but it won't run.

If you want to check out how this first run of my talk goes (as usual, I don't really rehearse talks so this is as spontaneous, "fresh" and probably embarrassing as it gets on this tour), Microsoft Netherlands will have a live webcast tomorrow that you can log into at http://www.microsoft.com/netherlands/msdn/devdays/webcast.asp.

Categories: Talks | EMEA Longhorn Preview | Indigo

January 20, 2004
@ 08:04 AM

I am getting ready for the Longhorn Developer Preview tour. Now that the whole notebook ordeal is hopefully over, I have been and still am polishing slides and we'll have an online rehearsal today during the day. Furthermore, we're working with Microsoft EMEA on a two day workshop about writing service oriented applications that consolidates all the thinking that I've been blogging about in the past year. The "sample" around which the workshop will center is, not very surprisingly, the FABRIQ.

I really need to get back into a "blogging mood".

To expect that the newest hardware works with a pre-Alpha version of the newest Microsoft operating system may be expecting a bit much. My Alienware Area51-m just wouldn't boot past the logo screen just 3 seconds after booting from the install disk. It just hung. Bummer.

To expect that a hardware vendor, especially one that's comparatively small and which is specialized in gaming machines and therefore very consumer focused, would even consider providing support on that issue is hopeless.

Is it? Well, usually it probably would be, but not with Alienware. Their tech support simply rocks. And with their help and help from the Longhorn Evangelism team in Redmond, Longhorn is now finally running on my new notebook.

The problem of the Area51-m not booting Longhorn is an unfortunate combination of a more BIOS-sensitive bootloader in Longhorn compared to XP/Win03 and a bug in current production AMIBIOS (AMIBIOS8, 1.09) that Alienware puts on their machines. Once we had that identified and I got the same fix that the Longhorn Evangelism team got for their Alienware machines (they have them too), flashed the BIOS and Longhorn booted.

Done? Unfortunately not. What I found was that this particular "special fix" BIOS version (1.08.01) would work stably with Longhorn and Win03 only when the machine is on AC power. Once you unplug and run on batteries, both OSses bluescreen after about 10-15 seconds.

Because this is my primary machine, I must have the machine running on batteries and therefore I re-flashed the BIOS back to the production version (1.09) so that at least Win03 would work and for Longhorn demos I'd just re-flash down to the other BIOS. Once done, I rebooted the machine and it happened to boot into Longhorn. And worked. Why would the installer hang so early on this BIOS version but the OS just boots fine once installed?  Puzzling.

So after all this had been sorted out, I figured that Longhorn isn't a good idea to have on the D: drive, after all. It does work, but I'd have to adjust a lot of demos and that's just too much work. So I am installing Win03 and Longhorn once more right now in the following sequence: BIOS 1.09 > Win03 to D: > "special fix" BIOS 1.08.1 > Longhorn to C: > BIOS 1.09. Now that we've got this sorted out, Alienware will hopefully have a permanent fix for the production BIOS soon so that this step becomes unnecessary and so that others can get Longhorn installed on their Area51s as well.

On the Longhorn tour, we'll have two of these boxes as our demo machines. Although they are absolutely swamped right now, Alienware made it possible to provide a system for Microsoft on very short notice, so that we don't have to carry a rather massive desktop PC around on "this 13 cities in 13 consecutive work days" tour as was initially planned.

Now I need to work on my backlog.

January 12, 2004
@ 05:58 PM

So my new notebook is an Alienware Area-51m. It's really, really fast, and looks great, but as of now, it doesn't get past the boot screen when I try to install Longhorn. The Longhorn boot screen starts fading in and the machine locks up. Win03 and XP work just fine ("great!" I should say). So I am sitting here, fiddling around with the install options and I am suspecting that something in the BIOS isn't quite like Longhorn expects it to be. To be continued ...

Blah, Blah ... "...if someone is using it who doesn't have a clue. Just like us."  Dismissing vulnerabilities in any operating system like that just to turn around and bash Microsoft while they're at it is just idiotic.

Categories: IT Strategy

January 8, 2004
@ 01:05 PM

No, it’s not that I am Russian-Orthodox (and it that case I’d be two days late, anyways). I am getting my purchased-specifically-for-Longhorn notebook tomorrow – finally. The manufacturer has kept us waiting for longer than they initially promised because apparently they’ve had some very strong demand for their boxes. So now I’ll have a very busy weekend installing stuff and I just hope it all works out. Specs: ATI Radeon 9600 / 128 MB graphics, 16.1” screen 1600x1200, 3.0GHz P4/HT w/ 800 MHz FSB, 1GB DDR RAM, 60GB HDD 7200 Rpm w/ 8MB cache, DVD+RW drive, 802.11 a/b/g wireless and then the usual stuff. That should be enough to do some cute and smooth Longhorn demos (and actual work).

Update: ... all that if DHL would be as quick to ship from Cologne to here (50km) as they are from Dublin to Cologne (where the package currently hangs out)

Categories: Other Stuff

January 6, 2004
@ 04:38 PM

Longhorn Developer Preview events all over Europe. I collected the event pages for you so you can register. The days in boldface are those on which I am speaking.

The Hague, The Netherlands: Developer Days 2004; Wednesday and Thursday, 21st/22nd January

Copenhagen, Denmark: Longhorn Developer Preview; Tuesday, 27th January

Helsinki, Finland: Longhorn Developer Preview; Wednesday, 28th January

Geneva, Switzerland: Developer Days 2004; Wednesday and Thursday, 28th/29th January

Budapest, Hungary: Longhorn Developer Preview; Friday, 30th January

Warsaw, Poland: Developer Days 2004; Monday and Tuesday, 2nd/3rd February

Portoroz, Slovenia: Developer Days 2004; Tuesday and Wednesday, 3rd/4th February

Dublin, Ireland:  Longhorn Developer Preview; Tuesday, 10th February

There are also events scheduled in Portugal (Feb 9), Spain (Feb 3), Croatia (Feb 5) and Italy (Feb 6), but I have yet to find the pages.

Categories: EMEA Longhorn Preview

January 6, 2004
@ 03:47 PM

Are you going to one of the Longhorn Developer Preview events or Developer Days 2004 events around Europe end of this month or beginning of February?

If you are, I have a question for you. I’ll be talking about Indigo and I am in a demo dilemma. My slide deck, like the ones that Steve, Joe, Don and the others showed at PDC, reflects mostly the status quo of the Indigo M5 milestone that is currently in development. The problem is that there are no M5 bits that I could show (there are not even M5 bits I could get). The PDC Longhorn build contains the Indigo milestone M4, whose programming model is very different – the “final” programming model for all of us happens to be added only in M5; whatever is in M4 is really an “internal” programming model that exists for testing purposes.

So what should I do? Spend more time explaining how things are going to be in the real thing (M5 and going forward into the Beta) or spend that time on doing M4 demos? Personally, I’d rather cut the demos entirely or show simply what needs to be done to get the SDK samples to run so that you have a starting point if you really want to play with the early bits. Make yourself heard; comment here.

Categories: EMEA Longhorn Preview

January 5, 2004
@ 09:07 AM

I’ve been terribly lazy in the last three weeks. Very little work. Meanwhile, I’ve completed the XIII game on XBOX which I got for Christmas and because I was in a “shoot and kill” mood anyways, I also finished about 2/3rds of Splinter Cell, which I got on my birthday, but never managed to play because I never took the time to do it. Now it’s back to business. Goodness. Happy 2004!

Categories: Other Stuff