(This post has been written by my coworker Eric Lam who pulled this all together – he wrote it as an email to the team – I’m just ripping that off)

NuGet (http://www.nuget.org/) is an open source package manager for .NET started by Microsoft. For some context there is a really good TechEd talk here.

1. Unless you have it (for instance as a side-effect of installing ASP.NET MVC3), get NuGet installed (it’s an VS extension) from here 
2. Right click on the project you want to add Service bus support to and select “Add Library Package Reference”

clip_image001

3. The NuGet manager will show up, and search for “AppFabric”, which will show all our AppFabric packages available right now (note: Cache is there too!)

clip_image002

4. Click install, and that’s it! The package do a number of things

  • Copy Microsoft.ServiceBus.dll and Microsoft.ServiceBus.Messaging.dll to your local project directories
  • Add the references to your project
  • If you have a App.Config and/or Web.Config, it will add the necessary WCF bindings/extensions for project consumption.

Some things to note for the package:

  • The packages require the projects to use the .Net 4.0 Full Profile (Client profile doesn’t work).
  • For sample code you will need to install it on a C# Console Application project.
  • This is only available for the CTP right now, not yet for the production SDK.

Let us know if you have any feedback or comments.

Categories: AppFabric

I’ve been starting to use Cinchcast to post little audio clips that I record while driving from and to work about things that come up with regards to Service Bus on the forums or on Twitter or elsewhere. The RSS feed is at http://www.cinchcast.com/clemensv.rss

Categories: Blog

Torek/Tuesday 24.5.

Uvodno predavanje za razvijalce: .NET and Windows Azure, a match made in the cloud – 08:30-09:30 – Europa A+C

Developer track keynote, together with Karl Davies-Barrett.

Windows Azure AppFabric: Building, Managing, and Connecting High-Density, Multi-Tenant Cloud Applications – 09:45-11:00 – Europa D 

Windows Azure AppFabric is Microsoft’s next-generation middleware application platform in the cloud, providing access control with federated identity, high-density, multi-tenant component-hosting, caching services, on-premise connectivity, rich publish/subscribe messaging, and integration services. In this session, Clemens Vasters, an Architect on the AppFabric product team at Microsoft will provide an overview of the AppFabric services that are already commercially available and the new services that Microsoft will bring to market until the end of this year.

Messages and Messaging – 15:45-17:00 – Emerald 2

As we were working on creating the API for Queues and Topics inside the Windows Azure AppFabric Service Bus team, we had a set of diverging opinions on what the message format should be. Some of us (Clemens, the speaker of this session, included) argued that the message format and object-model should be the same as the one for WCF – some argued that the message should be specific to Queues and Topics. As we were having those discussions it occurred to us that the same kind of debate was happening in synchronous messaging with the grand and protracted war of REST vs. SOAP. And as we started thinking about the parallels it because fairly clear that we were indeed having the exact same debate. In this session Clemens will discuss the findings, the outcome of our debate on the subject, and an analysis of why those debates even exist – which will hopefully enable you to make better decisions on what API abstraction level you want to buy into for your applications.

 

Sreda/Wednesday, 25.5.

Windows Azure AppFabric Service Bus: Messaging, Pub/Sub, and Connectivity in and through the Cloud – 16:00-17:00 – Europa C

Service Bus is the messaging and connectivity fabric of Microsoft’s Windows Azure AppFabric cloud-middleware platform. Service Bus provides endpoint-federation for Web services across network boundaries and topologies with NAT and Firewall traversal today and will, in the near future, also provide rich decoupled messaging services with publish/subscribe capabilities. In this presentation, Clemens Vasters, an Architect on the AppFabric product team at Microsoft will provide an overview of Service Bus as it is currently available for commercial use and will provide a detailed outlook on the new Service Bus features that will become available over the course of this year.

Categories:

Here’s a (incomplete) snapshot of what’s out there in terms of material for the new Service Bus CTP:

  • First read the release notes where we provide a summary of what’s new and what changed and also point out some areas of caution on parallel installs of the CTP and production SDKs.  
  • You can get the SDK bits from here. Get the right set of binaries for your machine (x64 or x86) and the right set of samples (CS or VB) and definitely get the user guide. We will have a NuGet package shortly that will allow you integrating the Service bus assembly and all necessary config incantations straight into your apps without even having the SDK on your machine.
  • The reference docs are located here. This is a CTP and the documentation is likewise in CTP state, so there are some gaps that we try to fill.
  • My introduction to the CTP is on the new AppFabric blog here.
  • At the same location you’ll find David Ingham’s primer on Queues.
  • My TechEd talk on the new features is now posted on Channel 9.
  • We have a video series providing high-level overviews on Service Bus.
  • Neudesic’s Rick Garibay provides a community insider’s perspective on the new features. Matt Davey also likes what he sees.
  • The forums. Go there for questions or suggestions.

There’s more on the way. Let me know if you write a blog post about what you find out so I can link to it.

Categories: AppFabric

A lot of partners and customers we talk to are telling us that they think of Service Bus as one of the key differentiators of the Windows Azure platform because it enables customers to build and interconnect applications that reflect the reality of where things stand with regard to moving workloads to cloud infrastructures: Today and for years to come, applications and solutions will straddle desktop and devices, customer-owned and operated servers and datacenters, and private and public cloud assets.

After a decade and more of application integration and process streamlining, no line-of-business application is and should ever again be an island.

If applications move to the cloud or if cloud-based SaaS solutions are to be integrated into enterprise solutions for individual customers, integration invariably requires capabilities like seamless access to services and secure, reliable message flow across network and trust boundaries. Also, as more and more applications are federated across trust boundaries and are built to work for multiple tenants, classic network federation technologies such as VPNs are often no longer adequate since they require a significant degree of mutual trust between parties as they permit arbitrary network traffic flow that needs to be managed.

We just released a new Community Technology Preview that shows that we’re hard at work and committed to expand Service Bus into a universal connectivity, messaging, and integration fabric for cloud-hosted and cloud-connected applications – and we invite you to take a look at our Windows Azure AppFabric SDK V2.0 CTP – May Update and accompanying samples.

Service Bus is already unique amongst platform-as-a-service offerings in providing a rich services relay capability that allows for global endpoint federation across network and trust boundaries.

As of today, we’re adding a brand-new set of cloud-based, message-oriented-middleware technologies to Service Bus that provide reliable message queuing and durable publish/subscribe messaging both over a simple and broadly interoperable REST-style HTTPS protocol with long-polling support and a throughput-optimized, connection-oriented, duplex TCP protocol.

The new messaging features, built by the same team that owns the MSMQ technology, but on top of a completely new technology foundation, are (of course) integrated with Service Bus’s naming and discovery capabilities and the familiar management protocol surface and allow federated access control via the latest release of the Windows Azure AppFabric Access Control service.

Queues

Service Bus Queues are based on a new messaging infrastructure backed by a replicated, durable store.

Each queue can hold up to 100MB of message content in this CTP, which is a quota we expect to expand by at least an order of magnitude as the service goes into production. Messages can have user-defined time-to-live periods with no enforced maximum lifetime.

The size of any individual message is limited to 256KB, but the session feature allows creating unlimited-size sequences of related messages whereby sessions are pinned to particular consumers and therefore enabling chunking of payloads of arbitrary sizes. The session state facility furthermore allows transactional recording of the progress a process makes as it consumes messages from a session and we also support session-based correlation, meaning that you can build multiplexed request/reply paths in a straightforward fashion.

Queues support reliable delivery patterns such as Peek/Lock both on the HTTP API and the .NET API that help ensuring processing integrity across trust boundaries where common mechanisms like distributed 2-phase transactions are challenging. Along with that, we have built-in detection of inbound message duplicates, allowing clients to re-send messages without adverse consequences if they’re ever in doubt whether a message has been logged in the queue due to intermittent network issues or an application crash.

In addition to a dead-letter facility for messages that cannot be processed or expire, Queues also allow deferring messages for later processing, for instance when messages are received out of the scheduled processing order and need to be safely put on the side while the process waits for a particular message to permit further progress.

Queues also support scheduled delivery – which means that you can hand a message to the queue infrastructure, but the message will only become available at a predetermined point in time, which is a very elegant way to build simple timers.

Topics

Service Bus Topics provide a set of new publish-and-subscribe capabilities and are based on the same backend infrastructure as Service Bus Queues – and have all the features I just outlined for Queues.

A Topic consists of a sequential message store just like a Queue, but allows for many (up to 2000 for the CTP) concurrent and durable Subscriptions that can independently yield copies of the published messages to consumers.

Each Subscription can define a set of rules with simple expressions that specify which messages from the published sequence are selected into the Subscription; a Subscription can select all messages or only messages whose user-or system defined properties have certain values or lie within certain value ranges. Rules can also include Actions, which allow modifying message properties as messages get selected; this allows, for instance, selecting messages by certain criteria and affinitizing those messages with sessions or to stamp messages with partitioning keys, amongst many other possible patterns.

The filtered message sequence represented by each Subscription functions like a virtual Queue, with all the features of Queues mentioned earlier. Thus, a Subscription may have a single consumer that gets all messages or a set of competing consumers that fetch messages on a first-come-first-served basis.

To name just a few examples, Topics are ideal for decoupled message fan-out to many consumers requiring the same information, can help with distribute work across partitioned pools of workers, and are a great foundation for event-driven architecture implementations.

Topics can always be used just like Queues by setting set up a single, unfiltered subscription and having multiple competing consumers pull messages from the subscription. The great advantage of Topics over Queues is that additional subscriptions can be added at any time to allow for additional taps on the message sequence for any purpose; audit taps that log pre-processing input messages into archives are a great example here.

Access Control Integration

This new CTP federates with the appfabriclabs.com version of the Access Control service, which is compatible with the Access Control “V2” service that is in available commercially since April. The current commercially available version of Service Bus federates with Access Control “V1”.

The Service Bus API to interact with Access Control for acquiring access tokens has not changed, but we are considering changes to better leverage the new federation capabilities of Access Control “V2”.

Customers who are setting up access control rules for Service Bus programmatically will find that there are significant differences between the management APIs of these two versions of the Access Control service. The current plan is to provide a staged migration for customers with custom access control rules on their Service Bus namespaces; migration will be an option for some period of time when we will operate the V1 and V2 versions of the Access Control Service side-by-side. We will publish concrete guidance for this migration over the next several months with initial details coming this week here on this blog.

What Changed and What’s Coming?

We believe that providing these capabilities in the cloud – paired with the features we already have available in Service Bus – will open up a whole new range of possibilities for cloud-hosted and cloud-enhanced applications. We have seen amazing business solutions built on Service Bus and based on customer feedback we’re convinced that the addition of a fully featured set of message-oriented middleware capabilities will enable even more powerful solutions to be built. Our intention is to make all capabilities contained in this preview commercially available in the second half of 2011.

The load balancing and traffic optimization features for the Relay capability of Service Bus that were added in the PDC’10 CTP of Service Bus have been postponed and are no longer available in this CTP. However, “postponed” does not mean “removed” and we are planning on getting these features back into a CTP release soon. We’ve traded these features for capabilities that we expect will be even more important for many customers: Full backwards compatibility between the current production release of Service Bus and the new version we’re presenting in this CTP, even though we have changed a very significant portion of the Service Bus backend. We are committed to provide full backwards compatibility for Service Bus when the capabilities of this CTP go into production, including backwards compatibility with the Microsoft.ServiceBus.dll that you already have deployed.

To help customers writing apps on platforms other than .NET we will also release Java and PHP samples for the new messaging capabilities in the next few weeks. These samples will be versions of the chat client implemented in the Silverlight and Windows Phone chat samples included in the SDK for this CTP release.

Lastly, and most importantly, the purpose of a Community Technology Preview is to collect feedback from the community. If you have suggestions, critique, praise, or questions, please let us know at http://social.msdn.microsoft.com/Forums/en-US/appfabricctp/

You can also Twitter me personally at @clemensv and I’ll let the team know what you have to say.

For the Service Bus Team,

Clemens Vasters
Principal Technical Lead
Windows Azure AppFabric Service Bus

Categories: AppFabric

From: John Doe 
Sent: Thursday, May 12, 2011 3:10 AM
To: Clemens Vasters
Subject: What is the average network latency for the AppFabric Service Bus scenario?
Importance: High

Hi Clemens,

A rough ballpark range in milliseconds per call will do. This is a very important metric for us to understand performance overhead.

Thanks,
John


From: Clemens Vasters
Sent: Thursday, May 12, 2011 7:47 AM
To: John Doe
Subject: RE: What is the average network latency for the AppFabric Service Bus scenario?

Hi John,

Service Bus latency depends mostly on network latency. The better you handle your connections, the lower the latency will be.

Let’s assume you have a client and a server, both on-premise somewhere. The server is 100ms avg roundtrip packet latency from the chosen Azure datacenter and the client is 70ms avg roundtrip packet latency from the chosen datacenter. Packet loss also matters because it gates your throughput, which further impacts payload latency. Since we’re sitting on a ton of dependencies it’s also worth telling that a ‘cold start’ with JIT impact is different from a ‘warm start’.

With that, I’ll discuss NetTcpRelayBinding:

  1. There’s an existing listener on the service. The service has a persistent connection (control channel) into the relay that’s being kept alive under the covers.
  2. The client connects to the relay to create a connection. The initial connection handshake (2) and TLS handshake (3) take about 5 roundtrips or 5*70ms = 350ms. With that you have a client socket.
  3. Service Bus then relays the client’s desire to connect to the service down the control channel. That’s one roundtrip, or 100ms in our example; add 50ms for our internal lookups and routing.
  4. The service then sets up a rendezvous socket with Service Bus at the machine where the client socket awaits connection. That’s just like case 2 and thus 5*100ms=500ms in our case. Now you have an end-to-end socket.
  5. Having done that, we’re starting to pump the .NET Framing protocol between the two sites. The client is thus theoretically going to get its first answer after a further 135ms.

So the handshake takes a total of 1135ms in the example above. That’s excluding all client and service side processing and is obviously a theoretical number based on the latencies I picked here. You mileage can and will vary and the numbers I have here are the floor rather than the ceiling of relay handshake latency.

Important: Once you have a connection set up and are holding on to a channel all subsequent messages are impacted almost exclusively by the composite roundtrip network latency of 170ms with very minimal latency added by our pumps. So you want to make a channel and keep that alive as long as you can.

If you use the Hybrid mode for NetTcpRelayBinding and the algorithm succeeds establishing the direct socket, further traffic roundtrip time can be reduced to the common roundtrip latency between the two sites as the relay gets out of the way completely. However, the session setup time will always be there and the Hybrid handshake (which follows establishing a session and happens in parallel) may very well up to 10 seconds until the direct socket is available.

For HTTP the story is similar, with the client side socket (not the request; we’re routing the keepalive socket) with overlaid SSL/TLS triggering the rendezvous handshake.

I hope that helps,
Clemens

Categories: AppFabric