With the March CTP, the .NET Service Bus namespace root for each project is taking on a new form that we had already pre-announced into the PDC’08 documentation and that I’ve talked about at PDC and other occasions. Any project’s root URI is now, depending on the chosen transport option, of the form

sb://project-name.servicebus.windows.net/ or
http://project-name.servicebus.windows.net/ or
https://project-name.servicebus.windows.net/

The previous root URI for a project was sb:|http:|https://servicebus.windows.net/services/project-name/, which was clearly sub-optimal for a number of reasons. Some technical, some more philosophical. Technical reasons first:

  • From the customer perspective, the old structure didn’t allow any project to host “magic” well-known files at the root of the project’s domain. There are unfortunately some protocols that require this, even though it’s broadly considered to be bad practice to bake particular assumptions of the location of resources into protocols. Amongst the “offenders” are robots.txt, favicon.ico and w3c/p3p.xml, but also Adobe Flash’s cross-domain policy crossdomain.xml and – taking a page out of Adobe’s playbook in that case – Microsoft’s Silverlight with ClientAccessPolicy.xml. Therefore the Service Bus was inherently Flash and Silverlight incompatible, unless you served up the apps from within the Service Bus itself. I’m quite sure that there are numerous proprietary protocols used with all sorts of applications that follow a similar pattern and so far those applications could not be exposed through the .NET Service Bus, because there wasn’t any way to put anything at the root. Now there is, even though it requires a little WCF trick that I’ll explain in a separate post specifically addressing crossdomain.xml and ClientAccessPolicy.xml.
  • The most important reason for us was that we need to be able to scale out the system into an “infinite” number of partitions (or “scale-units”) inside and across data-centers. Therefore, each project now gets its very own DNS record pointing to the scale-unit(s) where the project is hosted. The DNS system we’re leveraging is the one that’s used across most Windows Live properties and has a number of very interesting characteristics that will allow us – over time - to optimize the Service Bus traffic flow and to drive down end-to-end latency.

What matters even more to us from an “aesthetics” and design perspective is that we really want a project’s namespace to be clean, isolated, and want to avoid any artificial, system imposed pollution in the namespace. The fact that there are sound technical reasons supporting that desire is even more helpful to create consensus around this.

At its core, the Service Bus namespace is a federated, hierarchical service registry, whose structure is dictated and owned by “the project”. The difference between the Service Bus namespace and a “classic” service registry system like DNS or UDDI or LDAP is that services or messaging primitives are (usually) not only referenced by the registry, but they are projected straight into the registry, so that you can interact with the registry and those services or messaging primitives projected into the registry using similar or identical programming interfaces and within the scope of a unified namespace. We intentionally blur the distinction.

The service registry’s “discovery” (or “browsing”) protocol is based on a simple, conceptually infinite, nested hierarchy of ATOM 1.0 feeds. (In case you are wondering: Yes, we’ve got work lined up to provide flattened, non-hierarchical discovery overlaid over the hierarchy.) If you project an ATOM 1.0 endpoint into any name in the name hierarchy and choose to make that endpoint discoverable, the transition from the discovery hierarchy across the Service Bus namespace into the discovery of the resource hierarchy provided by your endpoint is absolutely seamless. That’s a design point.

What makes the namespace “federated” is that services or messaging primitives can be projected into the shared namespace from “anywhere”. Typically, the path portion of a URI represents a set of relatively tightly collocated set of resources that are residing across a web farm or a database cluster with the authority portion identifying (directly or indirectly) the target cluster.

.NET Service Bus URIs obviously look exactly like that, but they are quite different.

Imagine you have a distributed setup with 3 different “order processing” systems: One for the U.S. near Seattle, one for the EU near Düsseldorf, and one for the SEA region in Singapore.  Let’s also assume that I’m not talking about a multi-national mega-corporation here, but about a trading company with some 40 people that happens to have these three offices. Let’s also assume that they are using a customized ISV application that has been adapted for them by a (plug!) Microsoft Certified Partner and that application is primarily designed to run on local servers. Let’s also assume that it would be difficult (or prohibitively expensive) for them to get a static IP address and a corresponding, secure network setup that would allow them to wire up the application at the respective sites to the outside world. If you are helping customers with business apps, you might find that scenario to be resonating with your experience.

The .NET Service Bus can help with the connectivity issues by allowing to project the endpoints into the Service Bus namespace. That means that the application’s endpoints are listening for messages on and inside the .NET Service Bus instead of some local network listener on-site. There is no need to open any inbound firewall port and no need to do anything to the NAT setup and no need to do anything with DNS. Clients talk to those endpoints. The Service Bus namespace helps with getting those applications organized in a way that you can look at the resulting distributed system as “one” even though it spans sites:

http://littletradingcorp.servicebus.windows.net/orders/seattle/
http://littletradingcorp.servicebus.windows.net/orders/dusseldorf/
http://littletradingcorp.servicebus.windows.net/orders/singapore/

In combination with the .NET Access Control service, you can now overlay a single set of access control rules over the base scope http://littletradingcorp.servicebus.windows.net/orders/ which yields a centrally manageable access control overlay over these three services, even though the actual servers and endpoints are spread once around the world.    

What makes the naming system very different from DNS is that the .NET Service Bus naming system names endpoints and not hosts. Let’s say that each site also hosts a local “human resources” software; at that company size that may very well be an application that runs on the respective branch manager’s desktop machine or on a small server. That system is quite naturally distinct from from the order processing system and its reasonable safe to assume that the company wouldn’t want to collocate that system with the order processing system. Let’s project these into the namespace as well and we’ll certainly assume they have different Access Control rules that apply to the respective root scope:

http://littletradingcorp.servicebus.windows.net/hr/seattle/
http://littletradingcorp.servicebus.windows.net/hr/dusseldorf/
http://littletradingcorp.servicebus.windows.net/hr/singapore/

If we were trying to provide direct access to the “orders” and “hr” endpoints using HTTP with a “normal” setup, we would either have to have – especially with HTTPS – two public, static IP addresses for each site that are mapped to the respective machines or we’d have to use some gateway server that would dispatch the requests locally based on a host-header (requiring distinct DNS entries) or on the path prefix or we’d have to resort to non-standard ports – and we’d open up the floodgates for arbitrary and potentially malicious inbound traffic on the opened ports. And if we had that we would have to map these IP addresses into some DNS naming structure.  That’s quite a bit of networking work. Not necessarily complicated for someone who is well versed in these matters, but at the very least it’s a significant cost point. The resulting complication grows with each endpoint, since DNS identifies the public IP gateway and not the endpoint. So things get trickier and trickier. If we want to help small and medium businesses to “go digital” and intercommunicate more efficiently over the web (Fax is still the king in very many places and businesses), all those networking acrobatics don’t scale well.

Mind that I argue that not all business owners are happily accepting the notion of putting all their data and apps into someone else’s data center or "into the cloud”. Mind also that I don’t think I’m contradicting myself here. The .NET Service Bus is the “+” in what Microsoft calls “Software+Services”. It facilitates communication between places of which either can be in the cloud or at a customer-controlled site, it’s not the place where you “put data”.

So much for this post – in the next we’re going to look at the new March 2009 CTP Routers and how they interact and integrate with the namespace structure. 

Categories:

Our deployment team reports that they’re done with the last touches on our new release, that the servers are happily humming with the new bits, and that the new SDK is posted.

The .NET Services Developer Center has been updated as well. You can get the SDK bits from here [1].  We also have a runtime-only redistributable package that sits here [2].

What changed? Quite a bit. I’ll be updating you about the concrete changes in the .NET Service Bus beyond what we’ve got in the docs here on my blog over the next few days and also go into detail on some of the design considerations for the newly added functionality and the service APIs, specifically also on the significantly expanded REST capabilities.

People who’ve been following .NET Services or even BizTalk Services for a while will immediately notice that we’ve introduced a number of breaking changes in this release and that your existing applications will require some changes. The most important change we’ve made across the services and .NET Service Bus in particular is that we’re now giving every project a clean “root” URI:

  • Before the March CTP, the “root” of your namespace resided at: http://servicebus.windows.net/services/project-name/
  • With the March CTP, we’ve moved the project-name into the DNS name and drop the ‘services’ segment: http://project-name.servicebus.windows.net/

With that, we’re giving you complete control over your Service Bus namespace and we’re making every effort to leave you in the driver’s seat on the namespace design and to avoid hijacking URIs for “magic” functionality. What that means – and I will elaborate on that topic be showing you some code in the next few days – is that you can now do things like putting virtual robots.txt or Silverlight ClientAccessPolicy.xml or Flash crossdomain.xml endpoints at the root, enabling cross-domain scripting against relayed HTTP services.

The good news is that we’ve made no visible changes to the WCF bindings for the .NET Service Bus that we talked about at PDC. We’ve improved some of them in significant ways under the hood, but we didn’t touch the API surface in any way you ought to notice in your apps. A very important set of bug fixes that went in in that area is that we’re now respecting all size quotas and timeouts you set on the bindings. We didn’t really do a good job there in previous releases. When you ask for a 10 second “open” timeout for a binding that’ll actually be effective now. Before you were stuck with a minute no matter what you set the values to.

The “flagship” WCF binding of the .NET Service Bus, the NetTcpRelayBinding got the most attention. The binding knows how to do a particularly awesome networking stunt when you set the ConnectionMode=TcpConnectionMode.Hybrid. It will attempt (and mostly succeed) snapping a direct socket between two parties even through both parties may be hidden behind opposing firewalls and NATs. As we’ve been analyzing some bugs in this latest milestone we’ve found that the established connections were only working well for one-way communication, but not for full-duplex or even request-reply conversations. We’ve fixed that. What we’ve also added is a status indicator by which you can tell whether the connection is in “relayed” mode or whether it has been upgraded to “direct” mode. The new mechanism is shown in the Direct Connect sample, but I’ll quote that here for you as I go through the features in more detail.

A commonly seen customer issue is that quite a few people don’t have any leverage to get the outbound TCP ports that we require for outbound communication opened on an upstream firewall (remember: no inbound open firewall ports required; you can stay stealthy). We’ve started to address this across all our bindings so that the .NET Service Bus client and listener endpoints are a bit more aggressive in trying to get out to the .NET Service Bus. That particular feature is in the bits, is enabled by default and we expect (hope) that people who are trying to connect from within tightly managed network environments with outbound port restrictions will see things “just work”. We’re eager to find out whether people who’ve been reporting DNS resolution errors for servicebus.windows.net or nothing but obscure 1-minute timeout failures are finding that things have improved for them – albeit connections may not be as snappy as without such port restrictions in place. 

Absolutely new in this release are a set of composable communication primitives that I’ve talked about and demoed at MIX: Queues and Routers. There’s a whole lot of philosophy behind the way those primitives are designed, what those primitives indicate about where we’re going with the .NET Service Bus, how they integrate into the namespace, and there’s a lot to say on how you can interact with them and compose them and therefore I’ll talk about the namespace as well as Queues and Routers in detail and in a few separate posts starting later today.

[1] http://www.microsoft.com/downloads/details.aspx?FamilyID=8d1d1d5e-1332-4186-b33f-26d053759e49&displaylang=en
[2] http://www.microsoft.com/downloads/details.aspx?FamilyID=4eff38b3-fca0-4940-a014-461631152e5d&displaylang=en

Categories: