Teaching Indigo to do REST/POX, Part 1
A not so long time ago in a land far away…
A little bit more than half a year ago I got invited to a meeting at Microsoft in Redmond and discussed with Steve Swartz, Yasser Shohoud and Eugene Osovetsky how to implement POX and REST support for Indigo. You could also say that Steve dragged me into the meeting, since I happened to be on campus anyways and was burning some time in Steve’s office. I am not sure whether I made any good contribution to the cause in the meeting, but at least I witnessed the definition a special capability for the HTTP transport that I am exploiting with a set of Indigo extensions that I’ll present in this series of blog posts. The consensus in the meeting was that the requirements for building POX/REST support into the product weren’t generally clear enough in the sense that when you ask 100 people in the community you get 154 ever-changing opinions about how to write such apps. As a consequence it would not really be possible to define a complete programming model surface that everyone would be happy with, but that a simple set of hooks could be put into the product that people could use to build programming models rather easily.
And so they did, and so I did. This new capability of the HTTP transport first appeared in the September CTP of Indigo/WCF and surfaces to the developer as properties in the Message class Properties collection or the OperationContext.Incoming/OutgoingMessageProperties.
If you are using the Indigo HTTP transport on the server, the transport will always stick a HttpRequestMessageProperty instance into the incoming message properties, which provides access to the HTTP headers, the HTTP method (GET, POST, PUT, etc.) and the full query string. On the client, you can create an instance of this property class yourself and stick it into any outbound message’s properties collection and, with that, control how the transport performs the request. For sending replies from the server, you can put a HttpResponseMessageProperty into the message properties (or, again, into the OperationContext) and set the HTTP status code and description and of course the HTTP reply headers.
And since I have nothing better to do, I wanted to know whether this rather simple control feature for the HTTP transport would indeed be enough to build a POX/REST programming model and application when combined with the rest of the Indigo extensibility features. Executive Summary: Yes.
Now, the “yes” doesn’t mean that the required extensions write themselves in an hour. It’s a bit of work to get there, so it’ll take me a bit of writing (and a few days) to tell the whole story. There are two ways to approach the explanation: “Bottom-up” where I’d start at the wire level and show how to have Indigo send and accept any (and I mean any: 8GB+ video/mpeg files included) stuff that isn’t wrapped in SOAP envelopes or “Top-Down” where I start at the programming model surface and explain the application-level developer experience first. It’s a tough call, but I’ve decided to start with the latter.
REST and POX
In a nutshell – and I am sure that I’ll now get 153 corrections by comments and email from the other 99 people – the core ideas behind REST (representational state transfer) are that it builds on the pervasive HTTP web-architecture, that every item (“resource”) in the whole wide world can/could be identified by a HTTP URL and that HTTP has sufficient built-in methods to manipulate such resources. You can GET a data representation of the item, you can POST a new, related resource representation underneath an existing resource, you can PUT updates to resource representations and you can of course DELETE a resource representation. Every representation of a resource may have links to related resources that you can likewise access using these methods and therefore you get a web of information. REST is an architectural generalization of the HTML/HTTP web and the proponents of REST argue that the web works very obviously very well and that therefore REST is fit for any type of application. Of course, that’s a bit of an exaggeration, because there are plenty of scenarios where request/response isn’t the thing to do, but for a lot of applications REST may just be a good choice. Commonly, the REST architectural style is combined with XML data representations, even though this architectural style is really content-neutral. The place to find a “link” in a resource represented by a JPEG photo could very well be a URL on somewhere an advertising poster in the background behind your mother-in-law wearing a horrible hat at an English horse-racing event.
POX means “plain old XML” and I’ve also heard a definition saying that “POX is REST without the dogma”, but that’s not really correct. POX is not really well defined, but it’s clear that the “plain” is the focus and that means typically that folks who talk about “POX web services” explicitly mean that those services don’t use SOAP. You could see POX as an antonym for SOAP, if you will.
Except for “this isn’t SOAP” the term POX means very little. POX isn’t about architecture, it’s only about content. But REST is about architecture and it is content neutral. So what you commonly find is the combination of REST/POX as an alternative model to SOAP and the WS-* web services stack.
SOAP vs. REST/POX
Indigo is a messaging system that deals with messages that are represented by an envelope with headers and a body: SOAP. Indigo is so soaked with SOAP that you might just be able to wash your hands with a WinFX SDK CD. Still, the fact that Indigo is using an information model that is aligned with SOAP means nothing for what goes on or comes from the wire as I will show you later.
Fundamentally, the reason for why anybody might be using SOAP is that it gives you two places to stick things: A header section for metadata and a body section for payload. The headers are important if you need to communicate addressing, security or other information independent from a transport and across multiple communication hops. HTTP does have an extensible headers model, but other messaging transport options don’t. So to make things consistent across all sorts of transports and to provide an abstraction away from the transport specifics, all metadata needed to establish communication is stuck into an envelope alongside the payload. If you look at SOAP and WS-Addressing combined, you can find that the information content is really not much more than that of an IP packet (yes, IP=Internet Protocol). SOAP/WS-Addressing provide an abstraction for routing packages over any sort of transport just as much as IP provided an abstraction over Ethernet, Token Ring, ArcNet and so forth The problematically-named WS-ReliableMessaging is the TCP equivalent, by the way.
But: If you’ve decided that HTTP is all you need and REST/POX is the way to go for your application, you are apparently happy with a request/response model, you don’t need routing or reliable delivery, transport/app-protocol level security is sufficient, and you don’t need to have an abstraction of the HTTP headers and HTTP methods. In that case, the features that SOAP and the WS-* stack give you could be considered redundant. Let’s assume that’s so and try to eliminate SOAP out of the equation.
Figuring out a programming model
Incoming SOAP messages can be dispatched to handlers, typically methods, in two (and a half) different ways. The first (and a half) and most common option is to dispatch based on the value of the SOAPAction: (SOAP 1.1) HTTP header or the “action” media type parameter (SOAP 1.2). Alternatively, some stacks dispatch on the value of the WS-Addressing wsa:Action header, if present. The second way to dispatch messages it to look at the immediate child element of the soap:Body and to associate that content with a handler. The mapping rules are also spelled out in WS-Addressing. Indigo, like ASP.NET web services, uses the first method of dispatching, because that doesn’t require touching the message content.
Now, if we want to do away with SOAP and only want to kick plain old XML documents or even raw binary data around, we have a bit of a problem. Since there are neither any of the HTTP header hints, nor do we have a WS-Addressing wsa:Action header that Indigo could at, its got no information that it could use to dispatch the incoming request on. Even worse: If the incoming request is just an HTTP GET with no entity-body, at all, there’s really nothing to look at except, well, the HTTP method and the URI.
But let’s take a step back and look at this Indigo service contract (I spare you the WSDL) that makes any REST/POX person cringe and shout “This is RPC!”:
|
[ServiceContract] |
And, yes, this is very RPC like and I could use this exact contract with Enterprise Services or Remoting. It’s also perfectly fine for SOAP Web Services. The action values that Indigo uses for its dispatcher are, because we don’t specify any overrides in the attributes, derived from the method names combined with the port type name (here implicitly ICustomerInfo) and the contract namespace (here implicitly http://tempouri.org). So for this plain definition the action value for the operation GetCustomerInfo() is http://tempuri.org/ICustomerInfo/GetCustomerInfo. An incoming message with that exact URI as its action identifier is mapped to the operation implementation of the GetCustomerInfo() method.
However, without SOAP this dispatch strategy no longer works. Let’s assume from here on that the only thing that goes onto the wire are plain XML documents and not SOAP envelopes. We only send and receive XML payload with nothing around it. (You’ll get to see the “how to” later in this series).
Also, if you were a REST/POX person, you’d rightfully say that there is redundancy in this contract for two reasons. First, because HTTP is an application protocol (Mark Baker, one of the most visible REST proponents, keeps reminding the public about this) and it already defines methods for “Get”, “Update” (PUT), and “Delete”. With SOAP web services, everything is typically POSTed and so it’s effectively tunneling all sorts of semantics through what’s supposed to be the create semantics of HTTP and that makes purists fundamentally unhappy. Second, the “customerKey” for identifying the object is redundant because identifying the resource you want to modify or query is the job of the URL. (Note that I intentionally leave “Create” (POST) out of this for the moment. We’ll get back to that later.)
A more RESTish and HTTP aligned contract definition could look like this:
|
[ServiceContract] |
Now we assume, for a moment, that every customer in the system had its own HTTP service endpoint. If you have a million customers, you have a million endpoints, probably looking like this: http://www.example.com/myapp/customers/00212332. Each of these endpoints has an implementation of the shown interface, representing the resource.
Each of these million services already knows the customer key when a call reaches its endpoint, because each endpoint represents exactly one customer. Therfore we don’t have to pass any parameters to the service for Get or Delete. Get returns the data representation of the service endpoint’s very own, exclusively assigned customer. Following the same logic, the CustomerInfo record that we pass as an argument to Put() (Update) doesn’t need to contain an identifier. This new contract definition would also have sufficient metadata for a dispatcher extension for Indigo that we need to replace the SOAP action dispatcher, because we could map the incoming HTTP method directly to the respective handler by doing a (case-insensitive) string compare to the method name on the interface. HTTP GET maps to Get(), HTTP POST maps to Post(), and so forth.
I think that you probably even could create a million endpoints, but of course such an application would be a complete pig. I haven’t tried and I don’t think you should. So we should find a way to optimize the million endpoints down to one endpoint and from an implementation standpoint it would be rather useful if one service implementation could deal with multiple resource types. That means that we might want to have multiple GET or DELETE methods sitting on the same interface but handling different resources. That aside, “Post” and “Put” are not immediately intuitive names for “Create” and “Update” so that it’d be nice to decouple the application’s operation names from the HTTP method names.
To satisfy all of these requirements, I am adding a little additional metadata to this contract declaration:
|
[ServiceContract] |
The HttpMethodAttribute that I have written which I’ll show and explain in more detail in the next post, has a mandatory argument method which is the HTTP method that the operation handles. Along with that I’ll show an Indigo address filter and an Indigo operation selector that plug into the Indigo dispatch engine and perform the mapping for the incoming request to the correct handler based on the comparison of the HTTP method value and the attribute’s method value.
The optional, named attribute property UriSuffix allows narrowing the match to a namespace. If an implementation of this contract were hosted at http://www.example.com/myapp, only HTTP GET requests made on http://www.example.com/myapp/customers/ or any sub-path of that URL (the ‘*’ acts as wildcard) would be dispatched to GetCustomers(). Inside the method it is then very easy (String.LastIndexOf(‘/’)) to parse out the customer identifier from the URL that can be retrieved from the http request message property or the incoming message’s header collection (Indigo maps a set of HTTP transport information items to message headers if you ask it to). So if we were extending this service to also manage a set of phone/fax/mobile numbers for the customer, we could do it this way:
|
[ServiceContract] |
With the support for multiple namespaces we can create a neat, hierarchical external representation of our data whereby each data element has its individual URL. The fun part is that the application-level implementation does not differ greatly from what you would do in a “normal” app. The “magic” sits in the infrastructure. Sticking with our example, the home phone number data bit for a customer might be retrievable or manipulated here: http://www.example.com/myapp/customers/00212332/comm/home-phone.
Mind that for all examples I’ve shown here, I made the implicit suggestion that we’d use XML serialization. That’s really only for illustrative purposes. It turns out that a lot of the REST/POX proponents are also defenders of the pure beauty of angle brackets and therefore I’ll switch to a pure XML message model with the next post, because that is indeed quite a bit more flexible for this particular application style as you’ll see.