Part 1, Part 2, Part 3, Part 4
POX means “plain old XML” and I’ve also heard a definition saying that “POX is REST without the dogma”, but that’s not really correct. POX is not really well defined, but it’s clear that the “plain” is the focus and that means typically that folks who talk about “POX web services” explicitly mean that those services don’t use SOAP. You could see POX as an antonym for SOAP.
The design of Indigo (WCF) assumes that all messages that go onto the wire and come from the wire have a shape that is aligned with SOAP. That means that they have a “body” containing the user-defined message payload and a “header” section that contains the out-of-band metadata that helps getting the message from one place to the next, possibly through a chain of intermediaries. Most of the Indigo binding elements and their implementations also assume that those metadata elements (headers) conform to their respective WS-* standard that they are dealing with.
However, Indigo isn’t hard-wired to a specific envelope format. The default “encoders” that are responsible for turning a message into data stream (or a data package) that a transport can throw down a TCP socket or into a message queue (or whatever else) and which are likewise responsible for picking up the data from the wire to turn them into Message objects have two envelope formats baked in: SOAP 1.1 and SOAP 1.2. But that doesn’t mean that you have to use those. If your envelope format were different (there seem to be thousands, I’ll name AdsML [spec] as an example) and that’s what you want to use on the wire, you can assemble a binding that will compose an Indigo transport with your encoder. Moving away from SOAP means, though, that you can’t use the standard implementations of capabilities such as message-level security, reliable delivery, and transaction flow, because all of these are built on the assumption that you are exchanging WS-* headers with the other party and all of these specs depend on the SOAP information model. But if there are comparable specifications that come with your envelope format you can of course write Indigo extensions that you can configure into a binding just like you can compose the default binding elements. It’d be a lot of work to do that, but you’d still benefit greatly from the Indigo architecture per-se.
When we want to use a REST/POX model, our envelope format is quite simple: We don’t really have an envelope.
The idea of POX is that there’s only payload and that out-of-band metadata is unnecessary fluff. The idea of REST is that there is already and appropriate place for out-of-band metadata and that’s the HTTP headers.
In order to make REST/POX work, we therefore need to replace the Indigo default encoder with an encoder that fulfills these requirements:
1. Extract the message body XML content of any outbound message and format it for the wire as-is and without a SOAP envelope around it and
2. Accept an arbitrary inbound XML data and wrap it into a Message-derived class so that Indigo can handle it.
Since the use-case in whose context I’ve developed these extensions is a bit more far reaching than POX, but I indeed want to support RESTful access to any data including multi-GByte unencapsulated MPEG recordings I make on my Media PC, I’ve broadened these two requirements a bit and left out the “XML” constraint:
1. Extract the message body XML content of any outbound message and format it for the wire as-is and without a SOAP envelope around it and
2. Accept an arbitrary inbound XML data and wrap it into a Message-derived class so that Indigo can handle it.
XML aka POX is an interesting content-type to throw around, but it’s by no means the only one and therefore let’s not restrict ourselves too much here. Any content is good.
But then again, Indigo is assuming that all messages flowing through its channels contain XML payloads and therefore we’ve got a bit of a nut to crack when we want to use Indigo for arbitrary, non-XML payloads of arbitrary size. Luckily, XML is just an illusion.
The Indigo Message holds the message body content inside an XmlDictionaryReader (which is an optimized derivation of the well-known XmlReader). To construct a message, you can walk up to the static Message.CreateMessage(string action, XmlDictionaryReader reader) factory method and pass the readily formatted body content as a reader object and the message will happily adopt it. But can we use the XmlReader to smuggle arbitrary binary content into the message so that our own encoder can later unwrap it and put it onto the wire in whatever raw binary format we like? Sure we can! The class below may look a bit like an evil hack, but it’s a perfectly legal construct:
|
using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.IO;
using System.Xml.Schema;
namespace newtelligence.ServiceModelExtensions
{
public class PoxBase64XmlStreamReader : XmlTextReader
{
private const string xmlEnvelopeString =
"<base64Binary xmlns:xsi=\"" + XmlSchema.InstanceNamespace + "\" " +
"xmlns:xsd=\"" + XmlSchema.Namespace + "\" " +
"xsi:type=\"xsd:base64Binary\">placeholder</base64Binary>";
Stream innerStream;
/// <summary>
/// Initializes a new instance of the <see cref="T:PoxBase64XmlStreamReader"/> class.
/// </summary>
/// <param name="stream">The stream.</param>
public PoxBase64XmlStreamReader(Stream stream)
: base(new StringReader(xmlEnvelopeString))
{
innerStream = stream;
}
/// <summary>
/// Gets The Common Language Runtime (CLR) type for the current node.
/// </summary>
/// <value></value>
/// <returns>The CLR type that corresponds to the typed value of the node. The default is System.String.</returns>
public override Type ValueType
{
get
{
if (NodeType == XmlNodeType.Text && base.Value == "placeholder")
{
return typeof(Byte[]);
}
else
{
return base.ValueType;
}
}
}
/// <summary>
/// Gets the text value of the current node.
/// </summary>
public override string Value
{
get
{
if (NodeType == XmlNodeType.Text && base.Value == "placeholder")
{
BinaryReader reader = new BinaryReader(innerStream);
return Convert.ToBase64String(reader.ReadBytes((int)(reader.BaseStream.Length - reader.BaseStream.Position)));
}
return base.Value;
}
}
/// <summary>
/// Reads the content and returns the Base64 decoded binary bytes.
/// </summary>
/// <param name="buffer">The buffer into which to copy the resulting text. This value cannot be null.</param>
/// <param name="index">The offset into the buffer where to start copying the result.</param>
/// <param name="count">The maximum number of bytes to copy into the buffer. The actual number of bytes copied is returned from this method.</param>
/// <returns>
/// The number of bytes written to the buffer.
/// </returns>
public override int ReadContentAsBase64(byte[] buffer, int index, int count)
{
if (NodeType == XmlNodeType.Text && base.Value == "placeholder")
{
return innerStream.Read(buffer, index, count);
}
else
{
return base.ReadContentAsBase64(buffer, index, count);
}
}
}
} |
The PoxBase64XmlStreamReader is a specialized XML reader reading a fixed info-set constructed from a string that has a “placeholder” in whose place the content of a wrapped data stream is returned “as base64 encoded content”. Of course that latter statement is hogwash. The data is never encoded in base64 anywhere. But the consumer of the reader thinks that it is and that’s really good enough for us here. The XmlReader creates the illusion that the wrapped data stream were the “text” node of a base64Binary typed element and if that’s what the client wants to believe, we’re happy. The implementation trick here is of course very simple. As long as the reader isn’t hitting the text node with the “placeholder” all work is being delegated to the base class. Once we arrive at that particular node, we change tactics and return the data type (byte[]) and the content of the wrapped stream instead of the “placeholder” string. After that we continue delegating to the base class. If the client asks for the Value of the text node, we are returning a base64 encoded string representation of the wrapped stream which might end up being pretty big. However, if the client is a bit less naïve about the content, it will figure that the data type is byte[] and therefore retrieve the data in binary chunks through the ReadContentAsBase64() method. Let’s assume that the client will be that clever.
It doesn’t take too much imagination talent to do so, because I’ve got the client right here. I used Doug Purdy’s PoxEncoder that he showed at PDC05 as a basis for this and extended it (quite) a bit:
|
using System;
using System.IO;
using System.Xml;
using System.Text;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.ServiceModel.Design;
using System.Runtime.CompilerServices;
using System.ServiceModel.Configuration;
using System.Configuration;
using System.Globalization;
using System.Xml.Schema;
using System.Diagnostics;
namespace newtelligence.ServiceModelExtensions
{
/// <summary>
/// This class is a wire-format encoder for System.ServiceModel that renders
/// only the content (body) of a <see cref="T:Message"/> onto the wire, but not
/// the surrounding SOAP message elements such as the enevlope, the headers or
/// the body element. Likewise, the encoder expects input to be in 'raw', unwrapped
/// form and will wrap it into a message for processing by the System.ServiceModel
/// infrastructure.
/// </summary>
public class PoxEncoder : MessageEncoder
{
string contentType;
string mediaType;
Encoding textEncoding;
MessageVersion messageVersion;
/// <summary>
/// Creates a new instance of PoxEncoder
/// </summary>
public PoxEncoder()
{
messageVersion = MessageVersion.Soap11Addressing1;
textEncoding = Encoding.UTF8;
Initialize();
}
/// <summary>
/// Creates a new instance of PoxEncoder
/// </summary>
/// <param name="messageVersion"></param>
public PoxEncoder(MessageVersion messageVersion)
{
this. messageVersion = messageVersion;
textEncoding = Encoding.UTF8;
Initialize();
}
/// <summary>
/// Creates a new instance of PoxEncoder
/// </summary>
/// <param name="textEncoding"></param>
/// <param name="messageVersion"></param>
public PoxEncoder(Encoding textEncoding, MessageVersion messageVersion)
{
this. textEncoding = textEncoding;
this. messageVersion = messageVersion;
Initialize();
}
/// <summary>
/// Initializes common properties of the encoder.
/// </summary>
private void Initialize()
{
if (this.MessageVersion.Envelope == EnvelopeVersion.Soap12)
{
// set the aprorpiate media type for SOAP 1.2
this. mediaType = "application/soap+xml";
}
else if (this.MessageVersion.Envelope == EnvelopeVersion.Soap11)
{
// set the appropriate media type for SOAP 1.1
this. mediaType = "text/xml";
}
// compose the content type from charset and media type
this. contentType = string.Format(CultureInfo.InvariantCulture, "{0}; charset={1}", mediaType, textEncoding.WebName);
}
/// <summary>
/// Gets the content type for the encoder instance
/// </summary>
public override string ContentType
{
get
{
return contentType;
}
}
/// <summary>
/// Gets the media type for the encoder instance
/// </summary>
public override string MediaType
{
get
{
return mediaType;
}
}
/// <summary>
/// Gets an indicator for whether a given input content type is
/// supported.
/// </summary>
/// <param name="contentType">ContentType</param>
/// <returns>Indicates whether the content type is supported</returns>
/// <remarks>
/// TODO: This currently returns 'true' for all content types because the
/// encoder isn't locked down in features yet and this easier to debug.
/// The plan is to support at least: application/x-www-form-urlencoded,
/// text/xml, application/soap+xml
/// </remarks>
public override bool IsContentTypeSupported(string contentType)
{
return true;
}
/// <summary>
/// Gets the supported message version of this instance
/// </summary>
public override MessageVersion MessageVersion
{
get
{
return messageVersion;
}
}
/// <summary>
/// Reads an incoming array segment containing a message and
/// wraps it with a buffered message. The assumption is that the incoming
/// data stream is <i>not</i> a SOAP envelope, but rather an unencapsulated
/// data item, may it be some raw binary, an XML document or HTML form
/// postback data. This method is called if the inbound transfer mode of the
/// transport is "buffered".
/// </summary>
/// <param name="buffer">Buffer to wrap</param>
/// <param name="bufferManager">Buffer manager to help with allocating a copy</param>
/// <returns>Buffered message</returns>
public override Message ReadMessage(ArraySegment<byte> buffer, BufferManager bufferManager)
{
return new PoxBufferedMessage(buffer, bufferManager);
}
/// <summary>
/// Transforms an incoming message into a raw byte array that a transport can
/// literally put on the wire as it is returned. This method is called if the outbound
/// transfer mode of the transport is "buffered".
/// </summary>
/// <param name="msg">Input message</param>
/// <param name="maxMessageSize">Maximum message size to be rendered</param>
/// <param name="bufferManager">Buffer manager to optimize buffer allocation</param>
/// <param name="messageOffset">Offset into the message to render.</param>
/// <returns>Array segment containing the binary data to be put onto the wire by the transport.</returns>
/// <remarks>
/// <para>This method is the "secret sauce" of the the PoxEncoder. Instead of encoding the
/// message in its entirety, this encoder will unwrap the message body and toss out
/// the envelope and all headers. The resulting "raw" message body (everything inside
/// and not including soap:Body) will be written out to the transport.</para>
/// <para>The encoder has an optional, "out of band" argument that is flowing into it
/// as part of the message's Properties. By adding a <see cref="T:PoxEncoderMessageProperty"/>
/// to the <see cref="Message.Properties"/> and setting its <see cref="PoxEncoderMessageProperty.RawBinary"/>
/// property to 'true', you can switch the encoder into its 'raw binary' mode.</para>
/// <para> In 'raw binary' mode, the encoder expects that the only child of the message
/// body element is an element with a local name of "base64Binary" containing base64 encoded
/// binary data. If that is the case, the encoder will read the content of that element
/// and return it (not the XML wrapper) to the transport in binary form. If the content does
/// not comply with this requirement, an empty array is returned.
/// </para>
/// </remarks>
public override ArraySegment<byte> WriteMessage(Message msg, int maxMessageSize, BufferManager bufferManager, int messageOffset)
{
if (msg.IsEmpty)
{
// if the message is empty (no body defined) the result is an empty
// byte array.
byte[] buffer = bufferManager.TakeBuffer(maxMessageSize);
return new ArraySegment<byte>(buffer, 0, 0);
}
else
{
// check RawBinary bit in the message property
bool rawBinary = false;
if (msg.Properties.ContainsKey(PoxEncoderMessageProperty.Name))
{
rawBinary = ((PoxEncoderMessageProperty)msg.Properties[PoxEncoderMessageProperty.Name]).RawBinary;
}
ArraySegment<byte> retval = new ArraySegment<byte>();
byte[] buffer = bufferManager.TakeBuffer(maxMessageSize);
if (!rawBinary)
{
// If we're rendering XML data, we construct a memory stream
// over the output buffer, layer an XMLDictionaryWriter on top of it
// and have the message write the body content into the buffer as XML.
// The buffer is then wrapped into an array segment and returned.
MemoryStream stream = new MemoryStream(buffer);
XmlWriterSettings settings = new XmlWriterSettings();
settings.OmitXmlDeclaration = true;
settings.Indent = true;
settings.Encoding = this. textEncoding;
XmlWriter innerWriter = XmlWriter.Create(stream, settings);
XmlDictionaryWriter writer = XmlDictionaryWriter.CreateDictionaryWriter(innerWriter, false);
msg.WriteBodyContents(writer);
writer.Flush();
retval = new ArraySegment<byte>(buffer, 0, (int)stream.Position);
}
else
{
// If we're rendering raw binary data, we grab at most 'buffer.Length'
// bytes from the binary content of the base64Binary element (if that
// exists) and return the result wrapped into an array segment.
XmlDictionaryReader dictReader = msg.GetReaderAtBodyContents();
if (dictReader.NodeType == XmlNodeType.Element &&
dictReader.LocalName == "base64Binary")
{
if (dictReader.Read() && dictReader.NodeType == XmlNodeType.Text)
{
int size = dictReader.ReadContentAsBase64(buffer, 0, buffer.Length);
retval = new ArraySegment<byte>(buffer, 0, size);
}
}
}
return retval;
}
}
/// <summary>
/// Reads an incoming stream containing a message and
/// wraps it with a streamed message. The assumption is that the incoming
/// data stream is <i>not</i> a SOAP envelope, but rather an unencapsulated
/// data item, may it be some raw binary, an XML document or HTML form
/// postback data. This method is called if the inbound transfer mode of the
/// transport is "streamed".
/// </summary>
/// <param name="stream">Input stream</param>
/// <param name="maxSizeOfHeaders">Maximum size of headers in bytes</param>
/// <returns>Stream message</returns>
public override Message ReadMessage(System.IO.Stream stream, int maxSizeOfHeaders)
{
return new PoxStreamedMessage(stream, maxSizeOfHeaders);
}
/// <summary>
/// Transforms an incoming message into a stream that a transport can
/// literally put on the wire as it is filled. This method is called if the outbound
/// transfer mode of the transport is "streamed".
/// </summary>
/// <param name="msg">Input message</param>
/// <param name="stream">Stream to write to</param>
/// /// <remarks>
/// <para>This method is the "secret sauce" of the the PoxEncoder. Instead of encoding the
/// message in its entirety, this encoder will unwrap the message body and toss out
/// the envelope and all headers. The resulting "raw" message body (everything inside
/// and not including soap:Body) will be written out to the transport.</para>
/// <para>The encoder has an optional, "out of band" argument that is flowing into it
/// as part of the message's Properties. By adding a <see cref="PoxEncoderMessageProperty"/>
/// to the <see cref="Message.Properties"/> and setting its <see cref="PoxEncoderMessageProperty.RawBinary"/>
/// property to 'true', you can switch the encoder into its 'raw binary' mode.</para>
/// <para> In 'raw binary' mode, the encoder expects that the only child of the message
/// body element is an element with a local name of "base64Binary" containing base64 encoded
/// binary data. If that is the case, the encoder will read the content of that element
/// and write it (not the XML wrapper) onto the stream in binary form and in at most
/// 1MByte large chunks. If the content does not comply with this requirement, nothing is written.
/// </para>
/// </remarks>
public override void WriteMessage(Message msg, System.IO.Stream stream)
{
try
{
if (!msg.IsEmpty)
{
// check RawBinary bit in the message property
bool rawBinary = false;
if (msg.Properties.ContainsKey(PoxEncoderMessageProperty.Name))
{
rawBinary = ((PoxEncoderMessageProperty)msg.Properties[PoxEncoderMessageProperty.Name]).RawBinary;
}
if (!rawBinary)
{
// If we're rendering XML, we layer an XMLDictionaryWriter over the
// output stream and have the message render its body content into
// that writer and therefore onto the stream.
XmlWriterSettings settings = new XmlWriterSettings();
settings.OmitXmlDeclaration = true;
settings.Indent = true;
settings.Encoding = this. textEncoding;
XmlWriter innerWriter = XmlWriter.Create(stream, settings);
XmlDictionaryWriter writer = XmlDictionaryWriter.CreateDictionaryWriter(innerWriter, false);
msg.WriteBodyContents(writer);
writer.Flush();
}
else
{
// If we're rendering raw binary data, we grab chunks of at most 1MByte
// from the 'base64Binary' content element (if that exists) and write them
// out as binary data to the output stream. Chunking is done, because we
// have to assume that the body content is arbitrarily large. To optimize the
// behavior for large streams, we read and write concurrently and swap buffers.
XmlDictionaryReader dictReader = msg.GetReaderAtBodyContents();
if (dictReader.NodeType == XmlNodeType.Element && dictReader.LocalName == "base64Binary")
{
if (dictReader.Read() && dictReader.NodeType == XmlNodeType.Text)
{
byte[] buffer1 = new byte[1024*1024], buffer2 = new byte[1024*1024];
byte[] readBuffer = buffer1, writeBuffer = buffer2;
int bytesRead = 0;
// read the first chunk into the read buffer
bytesRead = dictReader.ReadContentAsBase64(readBuffer, 0, readBuffer.Length);
do
{
// the abort condition for the loop is that we can't read
// any more bytes from the input because the base64Binary element is
// exhausted.
if (bytesRead > 0 )
{
// make the last read buffer the write buffer
writeBuffer = readBuffer;
// write the write buffer to the output stream asynchronously
IAsyncResult result = stream.BeginWrite(writeBuffer, 0, bytesRead,null,null);
// swap the read buffer
readBuffer = (readBuffer == buffer1) ? buffer2 : buffer1;
// read a new chunk into the 'other' buffer synchronously
bytesRead = dictReader.ReadContentAsBase64(readBuffer, 0, readBuffer.Length);
// wait for the write operation to complete
result.AsyncWaitHandle.WaitOne();
stream.EndWrite(result);
}
}
while (bytesRead > 0);
}
}
}
}
}
catch
{
// the client may disconnect at any time, so that's an expected exception and absorbed.
}
}
}
}
|
The encoder shown above fulfills my two requirements and it is aware of the PoxBase64XmlReader trickery. It renders unencapsulated data onto the wire and accepts and wraps unencapsulated data from the wire. Furthernore, it supports buffered messages and it supports Indigo’s streaming mode, which allows sending messages of arbitrary size. What’s still missing in the picture is how we hook the encoder into the binding and how we can control whether the encoder works in “POX mode” rending XML or in “Raw Binary” mode rendering arbitrary data content. I might also have to explain what a PoxStreamedMessage is. I might also have to explain a bit better what the encoder does to begin with ;-)
Well, at least you have the code already, Part 6 comes with the prose.
