Finally, finally, finally. It was a looong wait. As many others, we were in a wait loop for WSE 2.0 for a long time and that let us do what we do today only much, much later than we initially anticipated. So after being able to test on and adjust for the WSE 2.0 RTM bits for the last four weeks, we're now happy enough with our "1.0" that we're ready to share it:

Microsoft EMEA and newtelligence AG present: The FABRIQ. (http://workspaces.gotdotnet.com/fabriq)
(When you go there, make sure you get both the bits and the Hands-on Labs; you will need them).

Also, a few things to keep in mind before you go and get the bits:

• This is a proof-of-concept project collaboratively created by Microsoft EMEA and newtelligence AG. We have tested intensively for quite a few sets of use-cases, but this is not a product. We are giving this to you because we think it's very useful architecturally and most implementation aspects isn't too bad either, and we do expect you to play with it. We don't give it to you to install this in a production environment tomorrow or even on the day after.
• The support policy for FABRIQ is very simple: There is none. If you download this, you are absolutely and entirely on your own, legally speaking. We are keen to hear your feedback and are curious whether and for what you find this useful, but this is no product and therefore there's no support whatsoever. (If you find this so useful that you want customization, support, or need help to get this from near-production quality to production-quality, sales@newtelligence.com is a great place to write e-mail to)
• This is "work in progress" and you are getting a version that is not considered finished. You will find artifacts in the code that are not (anymore or yet) used. You will find code branches that we not (anymore or yet) hit.  There are a few places, where we cut some corners in terms of implementation efficiency in order to get this out early. You will find that there is a bit of a disconnect between the specification documents that we have in the package vs. the documentation that you'll find and we could have done a better job cleaning it all up. We love this ourselves and will continue to polish it.
• You need WSE 2.0 and the Enterprise Instrumentation Framework to play.
• Contributions: We give you the code and you can use it and change it. For the first version and the next minor drops, we'll not have a public code repository that people can check things into immediately, because the beast turned out to be so complex that we need to stay in control for a little while. If we allowed "random" community contributions early, people who don't live inside in the codebase could too easily seemingly unrelated stuff. Therefore: If you want to change or add stuff, wrap up your changes along with a good reason why that's needed and send it here.
• Discussions: Write what you like or hate or what you don't understand into the forums in the workspace or just blog about it and refer to this entry or relevant entries on my blog or Arvindra's blog once he's fully set up. We'll accept everybody into the workspace; just apply and you'll be granted access as soon as someone sees it.

Credit where credit is due: Very many thanks to the development team in Argentina, with Eugenio Pace, Adrian Nigro, Federico Winkel, and Juan Carlos Elichirigoity, who have worked very very hard turning my "written in two weeks in a hurry" prototype code into something that's actually useful.

After providing some background on what nodes and networks are and how they work, I’ll get to how they are configured. Warning: This post is pretty dense in terms of content

It keeps happening to me and some of my good friends and it's getting to a point where something needs to be said:

Dear Clemens,

we are pleased to announce the YaddaYadda 2004 conference that will be held in Overseas City, Farwaway Country on Octember 24th-27th. We know your great presentation skills .. blah, blah ...

We will cover coach class airfare of up to $500 and will reserve you a room in the speaker hotel where we will cover the overnight stay and breakfast. We will also provide you with a free registration to attend all the sessions of YaddaYadda 2004.

YaddaYadda 2004 is a world-class conference with 40 speakers from 10 countries and we expect an audience of architects, developers and IT-managers from great companies so that we will give you a great platform and exposure to demonstrate your expertise.

Please let us know whether you are interested in speaking at YaddaYadda 2004 and submit your presenentations until ... blah, blah

Best Regards,

YaddaYadda Marketing Droid

The short answer is: No. For several reasons. First of all, YaddaYadda conference is one amongst a hundred conferences held each year in Faraway Country. You are not running a Microsoft TechEd or a Microsoft PDC where I meet a lot of my friends, and you are not running a high-profile academic event that would really interest me, you are running just another developer conference. So we have that out of the way now.

You know that I live in Germany, right? Sure you do. You know my name, you invite me, it should be pretty much public knowledge that I don't live in Faraway Country. So how do you expect me to even dare to talk to a travel agent about a ticket to Faraway Country for $500? Well, right, you say that if I book 12 weeks ahead of time on the cheapest possible connection (with a comfortable 8 hour layover in St. Someairport) I could get a flight for that price. True. Care to have a look at my schedule? I am Norway until 2 days before YaddaYadda 2004 and I need to go to South Africa from there. And as things happen, these dates may move. A tourist ticket that's inflexible and invalid unless I use those exact flights is practically worthless for anybody who needs to be as mobile as us guys (and gals) who are helping you guys out with content. And you know what? My travel agent is so good that he's getting a business class fare on the German gray market that's cheaper than any flexible economy fare that you could ever book from Faraway Country; go figure that out. And you know what? If other speakers decide to drive 20hrs to Overseas City, do you force them to come in a Yugo, too?

The free registration to your conference sounds like a nice benefit, but I'd like to decline that offer and rather trade those $1499 for cash. Of course it'd be a bit difficult for me to get to the session room without a badge, but I am sure you'll figure out how that works.

And just because you say that your conference will give me a great platform and exposure to demonstrate your expertise I suspect (no, I am sure) that you are not paying speakers, right? See, there's an immediate benefit to you and that's me and all the other speakers talking there and adding to the value of your conference. No, sorry, we are the value of your conference. I like to trade immediate benefit against immediate benefit and that usually either translates into $$$ or into something that's a bit more painful for you than giving out "exposure" - let's say a booth or an full page ad in one of your magazines plus a token of appreciation for the talk.

Do you want to know what the value of "exposure" is and how it translates into immediate business opportunities for the speaker? If you are lucky, you get a good lead out of one per 500 attendees (and really lucky if your get two or three) and that doesn't even guarantee a deal yet. How many people did you say will you have at your conference? 450? Sorry. Doesn't translate.

There are many reasons to speak at conferences. Some are just plain fun to be at. Some are great for the parties. Some are fantastic for the money (go figure). For speakers there are many motivations that range from "the kick" to speak in front of 800 people to liking themselves being on stage to actually earning their lives by speaking. Some of the speakers I know got so IPO lucky in 1999 that they're doing it just so and it has nothing at all with their business.

So how about some honesty and saying: We're throwing a conference, we're going to have a raving party at a bar at the beach and many of your best buddies are likely coming too. Minibar in the hotel is on us. Sorry, we can't pay you for speaking, because we're poor and need the money, but we can pay for a reasonable flight ticket and hotel and since we have this hotel deal anyways, why don't you just stay 3 days longer, use your air-miles to upgrade and maybe bring along your girlfriend? Now, YaddaYadda 2004, that'd be a lot more honest and if your location is cool enough I'd even come. Just don't try that exposure argument on me, please, Mr. Marketing Droid.

In this post, I describe the FABRIQ concepts of "networks" and "nodes":

The most fundamental element in FABRIQ is a message handler and handlers are organized in pipelines to process messages. I explain the relationship here.

We have one regular session:

• Architecture Overview Session (ARC405) with Arvindra Sehmi and myself: Wed, Jun 30 12:00 - 13:15 Room: 9b

along with a Hands-On-Lab and a Chalk-Talk 

• Internals Chalk Talk (CHT019) with Arvindra Sehmi (I will try to make it there. Thursday is very busy): Thu, Jul 1 10:15 - 11:30 Room: U
• Hands-On Lab (ARC-IL01) with newtelligence's Achim Oellers and Jörg Freiberger: Tue-Thu throughout the day, Room: O

For the impatient, this post shows two config snippets.

Before I can get into explaining how the FABRIQ works and how to configure it, I need to explain a bit of the terminology we use:

• A network is the FABRIQ term that's rougly equivalent to an "application". A network consists of an arbitrary number of network-distributed nodes that are running inside the scope of the network. The network creates a common namespace for all of these nodes. Networks are configured using a single XML configuration document that is submitted (or routed via another network) to all hosts that shall host the network's nodes.
• A node is the FABRIQ term that is roughly equivalent to a "service" or "component". A node is the smallest addressable unit. Every node has a "relative node URI" that is composed of the network name and the node's own name into {network/node}. This relative node URI can be composed with absolute, transport dependent URIs such as http://server/vdir/network/node or msmq://machine/queuename/network/node. Within a network, the runtime is also capable of resolving logical addresses of the form fabriq://network/node and automatically map them to physical addresses. At runtime, a node accepts messages and dispatches them into one of one or more action pipelines. Each node may be guarded by a set of WS-Policy assertions, including Kerberos and X.509 cert authentication and authorization claims. A node may be hosted on a dedicated machine, one a well defined set of machines or on "any" machine within a cluster.
• An action pipeline is a pipeline that is associated with an action identifier and is roughly equivalent to a "method". An action identifier is a URI as per WS-Addressing's definition of wsa:Action and is mapped to SOAPAction: whenever we go through HTTP. A node must host at least one action pipeline with no limit on the number of action pipelines it can support. An action may declare a set of message schema-types that it understands and those message definitions may be used for validation inbound messages. An action has one or more outbound message routes that are matched against the result message action or destination. Multiple routes may match a message, which causes the message flow to fork. For each route exist one or multiple prioritized routing destinations. If multiple destinations have the same priority, the engine will balance calls across those, otherwise the engine will use the ones with lower priority as backup routes. At the end of each action pipeline is a sender port that sends resulting messages out to their destinations, which may be other FABRIQ nodes or any other external endpoint that understands the respective one-way message being sent.
• A pipeline is a composition of a sequence of handlers or nested pipelines. Pipelines can be nested in arbitrary depth. Pipelines are strictly unidirectional message processors that have no concept of a "response" on the same thread analogous to a return value (hence all actions are one-way only). A pipeline may or may not be based on a predefinable pipeline-type. Pipeline-types allow the definition of reusable pipelines that can be reused within the same network or (via import) in multiple networks.
• A handler refers to a software component (a CLR class) implementing a set of interfaces that allow it to be composed into and hosted in a pipeline. Handlers should be designed to perform only very primitive operations that can then be composed into pipelines to implement specific functionality. Built-in handlers include a content-based routing handler and an XSLT transformation handler. Custom handlers may contain any type of logic. A handler receives messages and may consume them, evaluate and annotate them and yield any number of resulting messages. The definition of a handler embeds an XML fragment that allows the handler to configure itself. The actual reference to the CLR class implementing the handler is defined in a handler-type.
• A handler-type associates a CLR class with a name that can be used to define handlers within a configuration file. It also allows the declaration of a code-base URL for the CLR class. This feature allows the installation of "virgin" FABRIQ runtimes in a cluster and have the runtimes auto-download all the required code for hosting a node from a central code store and therefore dramatically eases deployment and dynamic reconfiguration of a FABRIQ cluster.

In the next couple of postings I will map these terms to concrete config files.

The interesting bit about config is that FABRIQ's configuration mechanism uses the FABRIQ itself. FABRIQ has a predefined (extensible, configurable) network "fabriq" with a node "configuration" that currently defines a single action "configure". The pipeline for that action consists of a single handler (the FabriqConfigurationHandler) and that expects and accepts the configuration files I'll describe over the next days as the body of a message. With that, the configuration mechanism can be secured with policy, or can be embedded into a larger network that does preprocessing or even performs automatic assembly of configuration, or that automatically distributes configuration from a single point across a large cluster of machines.

To be continued ...

Achim and myself are currently in a series of very quick rev-cycles for the first public release of the Microsoft/newtelligence FABRIQ project that we did with and for Microsoft EMEA HQ and that was conceived, driven and brillantly managed by my architect colleague Arvindra Sehmi, who gave me the lead architect role for this project.

[Reminder/Disclaimer: this is not a product, but rather a pretty elaborate "how-to" architecture example that comes with an implementation. Hence it's not a supported Microsoft or newtelligence "framework" or an attempt at some general, definitive guidance on how to write services. FABRIQ is an optimized architecture for fast, one-way, message processing within network-distributed nodes consisting of sequences of dynamically composed primitive processing steps. This isn't even trying to get anywhere near the guidance aspirations of Shadowfax, or let alone all the guidance we're getting from the Indigo team or even the parallel work I've been doing for MS by building Proseware.]

We've settled on build 1.0.4173 (yesterday) to be the TechEd version, but we still found a last minute issue where we weren't using WSE 2.0 correctly (not setting the SoapEnvelope.Context.Destination property for use with a bare WSE2 Pipeline in the presence of policy) and when I reassembled the distribution I didn't reset an option that I use for debugging on my machine and that caused installation hiccups over at Achim's machine. Achim commented the hour-long bug hunt with "Ah, you gotta love software!".

There will be hands-on labs at TechEd Europe led by Achim and Jörg that let you play with what we (very much including our friends at Microsoft Argentina and Microsoft EMEA) have built. And even if you don't have a proper use for a one-way queuing network architecture, it actually turned into a fun thing to play with. 

I'll be starting to explain aspects of the spec over the upcoming days and will explain how the architecture works, how you configure it and what its potential uses are. Already posted is some relevant information about the great idea of an XmlReader-based message design (which I designed inspired by the Indigo PDC build) and our use of lightweight transactions.

I am in the boot phase for the next software project right now (proprietary work) and I have identified very many good uses for the FABRIQ model in there already (hint).

Once all parties involved are giving their "thumbs up", we'll also make the source code drop and the binaries available to the public (you) and from there we're looking forward to your input (and contributions?).

I start to believe Mono is real. Check this out. The folks from the mono team apparently ported dasBlog to Mono's ASP.NET incarnation. My friend Martin Spedding gave me the "heads up". Thanks.

You read it here first. Kimberly Tripp blogs (rss). If you do anything with SQL Server: Subscribe!

Microsoft urgently needs to consolidate all the APIs that are required for provisioning services or sites. The amount of knowledge you need to have and the number APIs you need to use in order to lock down a Web service or Enterprise Services application programmatically at installation time in order to have it run under an isolated user account (with a choice of local or domain account) that has the precise rights to do what it needs to do (but nothing else) is absolutely insane. 

You need to set ACLs on the file system and the registry, you need to modify the local machine's security policy, you need to create accounts and add them to local groups, you must adhere to password policies with your auto-generated passwords, you need to conbfigure identities on Enterprise Services applications and IIS application pools, you need to set ACLs on Message Queues (if you use them), and you need to write WS-Policy documents to secure your WS front. Every single of these tasks uses a different API (and writing policies has none) and most of these jobs require explicit Win32 or COM interop. I have a complete wrapper for that functionality for my app now (which took way too long to write), but that really needs to be fixed on a platform level.