My grand boss ... if someone had told me this a year back ... but it turns out that it is a great blessing ... anyways .... My grand boss, the magnificient Doug Purdy points to our best kept secret: You can actually do Remoting-style distributed objects with WCF as Sowmy and Michael explain.

Update: Tomas Restrepo asks why that is good. Let me clarify: I think the transparent, distributed objects way of doing things is very problematic, but there are some scenarios where they are a feasible solution and there are migration scenarios where you don't have much of a choice. As a platform provider, we have a mainstream path (SO) that we prefer and that's represented in our turnkey scenarios, but we cannot and will not be as dogmatic as to shut the door on different architecture styles. We don't do that on REST/POX on one side and we don't do that on distributed objects on the other side of the spectrum.

The WinFX Tour is coming to Europe!

Mark it in your calendar and, if you can, sign up! Locations: Rotterdam (20 Apr), Nice (25 Apr), Zurich (2 May), Copenhagen (4 May), London (9 May), Eilat/IL (9 May), Reading/UK (10 May), Cairo (15 May), Moscow (19 May)

I'll be speaking at the Zurich, Copenhagen, and Eilat (TechEd Israel) events.

[If the event near you does not have a sign-up page linked, watch your local MSDN portal or MSDN newsletters for updates]

• You wrote an example for WCF that you want others to see?
• You wrote a WCF article on your blog or for a magazine? (online or offline, any programming and written language)
• You have a tool that complements or uses WCF? (any license, commercial and non-commercial)?
• You offer WCF training or speak at a conference?

I have the power to hyperlink. I want to know.

I am sure that some want to fly under our radar, but I am also sure that a lot of people are very interested to have a bit fat green spot showing up on our radar screen when it comes to their blogs posts. Well, if you look here ... everyone who left a comment on that post is on my blogroll in RSS Bandit and I am making every interesting and original post/thought/article visible internally to make sure that your wishes/concerns/praise are heard and your contributions to the community are acknowledged.

PS: Did I mention that I am involved in the MVP approval process?
PS: Identity (InfoCard, Active Directory, MIIS), Workflow and BizTalk gurus are welcome too. I will get your feed addresses to the right folks.

Today's news from Apple is significant. Sun already runs Windows and now Apple runs Windows. Cool.

Pablo Cibraro (who just received the Connected Systems Developer MVP award; Congratulations!) has built a compression channel for WCF.

Blogland is big. I am currently trying to get a bit of an overview what people out in blogland are doing with WCF. And while I've been doing that in addition to a bunch of very long and (due to the time difference between Redmond and Germany) very late evening meetings, Sabine has caught the Sudoku virus and keeps filling those grids ...

It turns out, there is convergence between WCF and Sudoku.

I have seen a few people pointing it out already, but in case you haven't seen Kumar Gaurav Khanna's WS-Sudoku (blog post) game, you might want to take a look. It's ClickOnce installable (given you have the WinFX Feb CTP) and lets a group of people solve a puzzle together. Very nice demo.

Mark, I care deeply about the hobbyist who writes some code on the side, the programmer who works from 9-5 and has a life and just as deeply about those who work 24/7 and about everybody in between

That said: now that we're getting close to being done with the "this vs. that" debate, we can most certainly figure out the "how can we optimize the programming experience" story. For very many people I've talked to in the past 4 years or so, reducing complexity is an important thing. I firmly believe that we can do enterprise messaging and Web-Style/Lo-REST/POX with a single technology stack that scales up and down in terms of its capabilities.  

Since I take that you are worried about code-bloat on the app-level, how would you think about the following client-side one-liners?

• T data = Pox.Get<T>("myCfg")
• T data = Pox.Get<T>("myCfg", new Uri("/customer/8929", UriKind.Relative));
• T data = Pox.Get<T>("myCfg", new Uri("http: //example.com/customer/8929"));
• T data = Pox.Get<T>(new Uri("http: //example.com/customer/8929"));
• U reply = Pox.Put<T,U>( new Uri("http: //example.com/customer/8929"), data, ref location));
• U reply = Pox.Post<T,U>( new Uri("http: //example.com/customer/"), data, out location));
• Pox.Delete(settings, new Uri("http: //example.com/customer/8929"));

Whereby "myCfg" refers to a set of config to specify security, proxies, and so forth; settings would refer to an in-memory object with the same reusable info. Our stack lets me code that sort of developer experience in a quite straightforward fashion and I can throw SOAPish WS-Transfer under it and make the call flow on a reliable, routed TCP session with binary encoding without changing the least bit.

If I am still missing your point in terms of ease of use and line count, make a wish, Mark. 

One of the things I've learned quickly on our team is that the customer is everything. That's not a marketing phrase but the literal truth. Up until January 31st I had great personal powers to cause 11th hour changes in WCF and since the day I joined up that power is gradually diminishing. That's not only because we're edging closer to RTM, but also because it's more difficult to fill in the "business justification" column for design changes that I would want to propose. There is a tension between "more and better features" and "shipping" and of course also a huge difference between a customer legitimately saying "I don't like that behavior" and "umm, so how do we make Rocky happy?".

It turns out that you (yes, you) have two very easy ways to make your suggestions heard and quite directly contribute to our product planning and to file bugs on things that don't work, things that you consider to be behaving in a strange way or stuff you plainly don't like or consider to be missing.

So if you think that we should have a [PatHelland] attribute that constrains the behavior of a WCF service to the exact guidance along the lines of Pat's Fiefdoms and Emissaries or Metropolis models we'd love to hear about it. (Even though the reply to that exact feature request would probably be an explanation of how to build that on top of the WCF extensbility model - you can actually build that attribute today 

1. The MSDN Forum. The forum is the place where all Program Managers on our team listen. We get a daily report of unanswered questions and we have an internal website where we can manage and assign the questions. So your questions do in fact land in our inboxes.

2. The MSDN Product Feedback Center. You can file bugs straight into our internal product database (called "Product Studio"). That tool is the most powerful way for anyone to submit bugs and feature requests. Whatever goes into the feedback center is an actual, unresolved product bug until it's been on the table and has been given serious consideration. We currently only have a tiny little number of bugs from the product feedback center in the database and we are of the humble opinion that we can't be that good

Filing bugs and suggesting features is always welcome. You shouldn't be worrying that we have a cutoff point for features at some point before RTM. That's our thing to do. There is always a next version and planning for that has actually started.

Inside the big house....

Back in December of last year and about two weeks before I publicly announced that I will be working from Microsoft, I started a nine-part series on REST/POX* programming with Indigo WCF. (1, 2, 3, 4, 5, 6, 7, 8, 9). Since then, the WCF object model has seen quite a few feature and usability improvements across the board and those are significant enough to justify that I rewrite the entire series to get it up to the February CTP level and I will keep updating it through Vista/WinFX Beta2 and as we are marching towards our RTM. We've got a few changes/extensions in our production pipeline to make the REST/POX story for WCF v1 stronger and I will track those changes with yet another re-release of this series.

Except in one or two occasions, I haven't re-posted a reworked story on my blog. This here is quite a bit different, because of it sheer size and the things I learned in the process of writing it and developing the code along the way. So even though it is relatively new, it's already due for an end-to-end overhaul to represent my current thinking. It's also different, because I am starting to cross-post content to http://blogs.msdn.com/clemensv with this post; however http://friends.newtelligence.net/clemensv remains my primary blog since that runs my engine

Listening

The "current thinking" is of course very much influenced by now working for the team that builds WCF instead of being a customer looking at things from the outside. That changes the perspective quite a bit. One great insight I gained is how non-dogmatic and customer-oriented our team is. When I started the concrete REST/POX work with WCF back in last September (on the customer side still working with newtelligence), the extensions to the HTTP transport that enabled this work were just showing up in the public builds and they were sometimes referred to as the "Tim/Aaaron feature". Tim Ewald and Aaron Skonnard had beat the drums for having simple XML (non-SOAP) support in WCF so loudly that the team investigated the options and figured that some minimal changes to the HTTP transport would enable most of these scenarios**. Based on that feature, I wrote the set of dispatcher extensions that I've been presenting in the V1 of this series and newtellivision as the applied example did not only turn out to be a big hit as a demo, it also was one of many motivations to give the REST/POX scenario even deeper consideration within the team.

REST/POX is a scenario we think about as a first-class scenario alongside SOAP-based messaging - we are working with the ASP.NET Atlas team to integrate WCF with their AJAX story and we continue to tweak the core WCF product to enable those scenarios in a more straightforward fashion. Proof for that is that my talk (PPT here) at the MIX06 conference in Las Vegas two weeks ago was entirely dedicated to the non-SOAP scenarios.

What does that say about SOAP? Nothing. There are two parallel worlds of application-level network communication that live in peaceful co-existence:

The Faceless Web

The first scenario is the web as we know it. Almost. HTTP is an incredibly rich application protocol once you dig into RFC2616 and look at the methods in detail and consider response codes beyond 200 and 404. HTTP is strong because it is well-defined, widely supported and designed to scale, HTTP is weak because it is effectively constrained to request/response, there is no story for server-to-client notifications and it abstracts away the inherent reliability of the transmission-control protocol (TCP). These pros and cons lists are not exhaustive.

What REST/POX does is to elevate the web model above the "you give me text/html or */* and I give you application/x-www-form-urlencoded" interaction model. Whether the server punts up markup in the form of text/html or text/xml or some other angle-bracket dialect or some raw binary isn't too interesting. What's changing the way applications are built and what is really creating the foundation for, say, AJAX is that the path back to the server is increasingly XML'ised. PUT and POST with a content-type of text/xml is significantly different from application/x-www-form-urlencoded. What we are observing is the emancipation of HTTP from HTML to a degree that the "HT" in HTTP is becoming a misnomer. Something like IXTP ("Interlinked XML Transport Protocol" - I just made that up) would be a better fit by now.

The astonishing bit in this is that there has been been no fundamental technology change that has been driving this. The only thing I can identify is that browsers other than IE are now supporting XMLHTTP and therefore created the critical mass for broad adoption. REST/POX rips the face off the web and enables a separation of data and presentation in a way that mashups become easily possible and we're driving towards a point where the browser cache becomes more of an application repository than merely a place that holds cacheable collateral. When developing the newtellivision application I have spent quite a bit of time on tuning the caching behavior in a way that HTML and script are pulled from the server only when necessary and as static resources and all actual interaction with the backend services happens through XMLHTTP and in REST/POX style. newtellivision is not really a hypertext website, it's more like a smart client application that is delivered through the web technology stack.

Distributed Enterprise Computing

All that said, the significant investments in SOAP and WS-* that were made my Microsoft and industry partners such as Sun, IBM, Tibco and BEA have their primary justification in the parallel universe of highly interoperable, feature-rich intra and inter-application communication as well as in enterprise messaging. Even though there was a two-way split right through through the industry in the 1990s with one side adopting the Distributed Computing Environment (DCE) and the other side driving the Common Object Request Broker Architecture (CORBA), both of these camps made great advances towards rich, interoperable (within their boundaries) enterprise communication infrastructures. All of that got effectively killed by the web gold-rush starting in 1994/1995 as the focus (and investment) in the industry turned to HTML/HTTP and to building infrastructures that supported the web in the first place and everything else as a secondary consideration. The direct consequence of the resulting (even if big) technology islands hat sit underneath the web and the neglect of inter-application communication needs was that inter-application communication has slowly grown to become one of the greatest industry problems and cost factors. Contributing to that is that the average yearly number of corporate mergers and acquisitions has tripled compared to 10-15 years ago (even though the trend has slowed in recent years) and the information technology dependency of today's corporations has grown to become one of the deciding if not the deciding competitive factor for an ever increasing number of industries.

What we (the industry as a whole) are doing now and for the last few years is that we're working towards getting to a point where we're both writing the next chapter of the story of the web and we're fixing the distributed computing story at the same time by bringing them both onto a commonly agreed platform. The underpinning of that is XML; REST/POX is the simplest implementation. SOAP and the WS-* standards elevate that model up to the distributed enterprise computing realm.

If you compare the core properties of SOAP+WS-Adressing and the Internet Protocol (IP) in an interpretative fashion side-by-side and then also compare the Transmission Control Protocol (TCP) to WS-ReliableMessaging it may become quite clear to you what a fundamental abstraction above the networking stacks and concrete technology coupling the WS-* specification family has become. Every specification in the long list of WS-* specs is about converging and unifying formerly proprietary approaches to messaging, security, transactions, metadata, management, business process management and other aspects of distributed computing into this common platform.

Convergence

The beauty of that model is that it is an implementation superset of the web. SOAP is the out-of-band metadata container for these abstractions. The key feature of SOAP is SOAP:Header, which provides a standardized facility to relay the required metadata alongside payloads. If you are willing to constrain out-of-band metadata to one transport or application protocol, you don't need SOAP.

There is really very little difference between SOAP and REST/POX in terms of the information model. SOAP carries headers and HTTP carries headers. In HTTP they are bolted to the protocol layer and in SOAP they are tunneled through whatever carries the envelope. [In that sense, SOAP is calculated abuse of HTTP as a transport protocol for the purpose of abstraction.] You can map WS-Addressing headers from and to HTTP headers.

The SOAP/WS-* model is richer, more flexible and more complex. The SOAP/WS-* set of specifications is about infrastructure protocols. HTTP is an application protocol and therefore it is naturally more constrained - but has inherently defined qualities and features that require an explicit protocol implementation in the SOAP/WS-* world; one example is the inherent CRUD (create, read, update, delete) support in HTTP that is matched by the explicitly composed-on-top WS-Transfer protocol in SOAP/WS-*

The common platform is XML. You can scale down from SOAP/WS-* to REST/POX by putting the naked payload on the wire and rely on HTTP for your metadata, error and status information if that suits your needs. You can scale up from REST/POX to SOAP/WS-* by encapsulating payloads and leverage the WS-* infrastructure for all the flexibility and features it brings to the table. [It is fairly straightforward to go from HTTP to SOAP/WS-*, and it is harder to go the other way. That's why I say "superset".]

Doing the right thing for a given scenario is precisely what are enabling in WCF. There is a place for REST/POX for building the surface of the mashed and faceless web and there is a place for SOAP for building the backbone of it - and some may choose to mix and match these worlds. There are many scenarios and architectural models that suit them. What we want is

One Way To Program. 

* REST=REpresentational State Transfer; POX="Plain-Old XML" or "simple XML"

Alien Abductions coming to MSDN. I have set up a cross-posting blog at http://blogs.msdn.com/clemensv and this post is for testing whether that works -- Cheers, Clemens

Below are the newtellivision bits for the February CTP that I am showing today at the MIX'06 conference.

This also contains the newest revision of the (BSD licensed) REST/POX Service Model extensions, which can used standalone.

Mind that the Windows Media Player experience has been thrown off its feet by an unintended side-effect of the recent KB911565 Windows Media Player security hotfix. So if you are running the WMP experience and you cannot click anything, temporarily uninstalling that hotfix will solve the problem. The issue is that script-generated <a onclick=""> tags in HTML will not fire the onclick event and I am using quite a few of those. Just haven't had to time to finda good workaround that doesn't make the HTML entirely awkward.

newtellivisionSetup-2006-03-20.zip (930.95 KB)

newtellivisionSource-2006-03-20.zip (701.65 KB)

Mind that the newtellivision application is for non-commercial use only. If you are interested in working with newtelligence regarding commercial licenses or commercialization in general, write email to info@newtelligence.com

I am putting the finishing touches on the next revision of newtellivision today. The code base, including a revision to my REST/POX extensions, is updated for the February CTP of WinFX/WCF and there are several fixes for the Windows Media Player experience.

I am keeping my fingers crossed that I will succeed showing recorded and live TV beamed straight from my home server here in Meerbusch, Germany (the number of possible peripheral points of failure regarding available bandwidth, network, routers, cable signal, etc. are astomishing, considering the "demo effect") at the MIX conference in Las Vegas, at my session at 3pm on Monday, March 20.

The updated code-base will be available before the session. The PowerPoint deck for the session is fun. There's lot see and hear. If you are coming to MIX or if you know someone who is coming to MIX come or tell them to go. Indigo, err, the Windows Communication Foundation is more committed to the "Web 2.0" story than you might know.

[Oh, and Germany is playing the USA in a football (soccer) friendly on Wednesday in Dortmund, Germany just when MIX winds down. That's precisely the use-case for my app ] 

I kicked off quite a discussion with my recent post on O/R mapping. Some people think I am completely wrong, some say that it resonates with their experience, some say I wrote this in mean spirit, some are jubilating. I particularly liked the "Architectural Truthiness" post by David Ing and the comment by "Scott E" in my comments section who wrote:

I've hiked up the learning curve for Hibernate (the Java flavor) only to find that what time was saved in mapping basic CRUD functionality got eaten up by out-of-band custom data access (which always seems to be required) and tuning to get performance close to what it would have been with a more specialized, hand-coded DAL.

As always, it's a matter of perspective. Here is mine: I went down the O/R mapping route in a project in '98/'99 when my group at the company I was working for at the time was building a new business framework. We wrote a complete, fully transparent O/R mapper in C++. You walked up to a factory which dehydrated objects and you could walk along the association links and the object graph would either incrementally dehydrate or dehydrate in predefined segments. We had filtering capabilities that allowed to constrain 1:N collections with large N's, we could auto-resolve N:M relationships, had support for inheritance, and all that jazz. The whole framework was written with code generation in mind. Our generators were fed with augmented UML class diagrams and spit out the business layer, whereby we had a "partial classes" concept where we'd keep the auto-gen'd code in one tree and the parts that were supposed to be filled manually in another part of the code tree. Of course we'd preserve changes across re-gen's. Pure OO nirvana.

While the platforms have evolved substantially in the past 7 years, the fundamental challenges for transparent (fully abstracted) mapping of data to objects remain essentially the same.

• Given metadata to do the mapping, implementing CRUD functionality with an O/R mapper is quite easy. We had to put lots of extra metadata into our C++ classes back in the day, but with .NET and Java the metadata is all there and therefore CRUD O/R mapping is very low-hanging fruit on both platforms. That's why there's such a large number of projects and products.
• Defining and resolving associations is difficult. 1:N is hard, because you need to know what your N looks like. You don't want to dehydrate 10000 objects to find a value in one of them or to calculate a sum over a column. That's work that's, quite frankly, best left in the database. I realize that some people worry how that leads to logic bleeding into the database, but for me that's a discussion about pureness vs. pragmatism. If the N is small, grabbing all related objects is relatively easy - unless you support polymorphism, which forces the mapper into all sorts of weird query trees. 1:N is so difficult because an object model is inherently about records, while SQL is about sets. N:M is harder.
• "Object identity" is a dangerous lure. Every object has its own identifier. In memory that is its address, on disk that's some form of unique identifier. The idea of making the persistent identifier also the in-memory identifier often has the design consequence of an in-memory "running object table" with the goal of avoiding to load the same object twice but rather linking it appropriately into the object graph. That's a fantastic concept, but leads to all sort of interesting concurrency puzzles: What do you do if you happen to find an object you have already loaded as you resolve an 1:N association and realize that the object has meanwhile changed on disk? Another question is what the scope of the object identity is. Per appdomain/process, per machine or even a central object server (hope not)?
• Transactions are hard. Databases are doing a really good job with data concurrency management, especially with stored procedures. If you are loading and managing data as object-graphs, how do you manage transaction isolation? How do you identify the subtree that's being touched by a transaction? How do you manage rollbacks? What is a transaction, anyways?
• Changing the underlying data model is hard. I've run into several situations where existing applications had to be, with the customer willing to put money on the table, be integrated with existing data models. O/R mapping is relatively easy of the data model falls out of the object model. If an existing data model bubbles up against an object model, you often end up writing a DAL or the O/R in stored procedures.
• Reporting and data aggregation is hard. I'll use an analogy for that: It's really easy to write an XPath query against an XML document, but it is insanely difficult to do the same navigating the DOM.

That said, I am not for or against O/R mapping. There are lots of use cases with a lot of CRUD work where O/R saves a lot of time. However, it is a leaky abstraction. In fact is is so leaky that we ended up not using all that much of the funkyness we put into our framework, because "special cases" kept popping up. I am pointing out that there are a lot of fundamental differences between what an RDBMS does with data and how OOP treats data. The discussion is in part a discussion about ISAM vs. RDBMS.

The number of brain cycles that need to be invested for a clean O/R mapping of a complex object model in the presence of the fundamental challenges I listed here (and that list isn't exhaustive) are not automatically less than for a plain-old data layer. It may be more. YMMV.

Now you can (and some already have) ask how all of that plays with LINQ and, in particular, DLINQ. Mind that I don't work in the LINQ team, but I think to be observing a subtle but important difference between LINQ and O/R*: 

• O/R is object->relational mapping.
• LINQ is relational->object mapping.

LINQ acknowledges the relational nature of the vast majority of data, while O/R attempts to deny it. LINQ speaks about entities, relations and queries and maps result-sets into the realm of objects, even cooking up classes on the fly if it needs to. It's bottom up and the data (from whatever source) is king. Objects and classes are just tooling. For O/R mapping, the database is just tooling.