I can't decide what I dislike more: IE6 randomly locking up or Firefox crashing on uncaught null pointer exceptions.
(1) Policy-negotiated behavior, (2) Explicitness of Boundaries, (3) Autonomy and (4) Contract/Schema Exchange are the proclaimed tenets of service orientation. As I am getting along with the design for the services infrastructure we're working on, I find that one of the four towers the others in importance and really doesn't really fit well with them: Autonomy.
P, E and CE say things about the edge of a service. P speaks about how a service negotiates its edge behavior, guarantees and expectations with others. E speaks about how talking to another service is and must be different from talking to anything within your own service. CE speaks about how it is possible to establish a common understanding about messages and data while allowing independent evolution of internals by not sharing programming-level type constructs.
P is about creating and using techniques for metadata exchange and metadata controlled dynamic configuration of service-edges, E is about creating and using techniques for shaping service-edge communication paths and tools in a way that services don't get into each others underwear, and CE is about techniques for defining, exchanging, and implementing the flow of data across service-edges so that services can deal with each other's "stuff".
P, E and CE are guiding tenets for "Web Services" and the stack that evolves around SOAP. These are about "edge stuff". Proper tooling around SOAP, WSDL, XML Schema, WS-Policy (think ASMX, WSE, Axis, or Indigo) makes or will make it relatively easy for any programmer to do "the right thing" about these tenets.
Autonomy, on the other hand, is rather independent from the edge of a service. It describes a fundamental approach to architecture. An "autonomous digital entity" is "alive", makes independent decisions, hides its internals, and is in full control of the data it owns. An autonomous digital entity is not fully dependent on stuff happening on its edge or on inbound commands or requests. Instead, an autonomous service may routinely wake up from a self-chosen cryostasis and check whether certain data items that it is taking care of are becoming due for some actions, or it may decide that it is time to switch on the lights and lower the windows blinds to fends off burglars while its owner is on vacation.
Autonomy is actually quite difficult to (teach and) achieve and much more a matter of discipline than a matter of tooling. If you have two "Web services" that sit on top of the very same data store and frequently touch what's supposed to be each others private (data) parts, each of them may very well fulfill the P, E, CE tenets, but they are not autonomous. If you try to scale out and host such a pair or group of Web services in separate locations and on top of separate stores, you end up with a very complicated Siamese-twins-separation surgery.
That gets me to very clearly separate the two stories: Web Services <> Services.
A service is an autonomous digital entity that may or may not accept commands, inquiries (and data) from the outside world. If it chooses to accept such input, it can choose to do so through one or more web service edges that fulfill the P,E,CE tenets and/or may choose to accept totally different forms of input. If it communicates with the outside world from within, it may or may choose to do so via web services. A service is a program.
A web service is an implementation of a set of edge definitions (policies and contracts and channels) that fronts a service and allows services to communicate with each other. Two services may choose to communicate using multiple web services, if they wish to do so. A web service is a communication tool.
With that, I'll cite myself from three paragraphs earlier: If you have two "Web services" that sit on top of the very same data store [...] they are not autonomous. If you try to scale out [...] you end up with a very complicated Siamese-twins-separation surgery." ... and correct myself: If you have two "Web services", they don't sit on the data store, they front a service. The service is the smallest deployable unit. The service provides the A, the web services bring P, E and CE to the table. A web service may, indeed, just be a message router, security gateway, translator or other intermediary that handles messages strictly on the wire-level and dispatches them to yet another web service fronting an actual service that does the work prescribed by the message.
All of this, of course, causes substantial confusion about the duplicate use of the word service. The above it terribly difficult to read. I would wish it was still possible to kill the (inapproprate) term "web service" and just call it "edge", or "XML Edge", or (for the marketing people) "SOAP Edge Technology", or maybe "Service Edge XML", although I don't think the resulting acronym would go over well in the U.S.
This Slashdot story reminds me why I hate NBC Sports. I lived in New York in 1996 during the Olympic Games and I got to see soapy stories about which terrible obstacles athletes had overcome to end up winning whatever competition and ... NO SPORTS. And in 2000 I happened to be in the U.S. again during the Olympics and I got to see NO SPORTS. German ARD/ZDF will have literally hundreds of hours of live coverage from Athens from early in the morning into the night each day. Olympics is about sports and it's also about the guy who comes in on 36th place in Marathon, running a record for his country. NBC dilutes the experience. too bad, America doesn't get to see the Olympics.
I don't blog much in summer. That's mostly because I am either enjoying some time off or I am busy figuring out "new stuff".
So here's a bit of a hint what currently keeps me busy. If you read this in an RSS aggregator, you better come to the website for this explanation to make sense.
This page here is composed from several elements. There are navigation elements on the left, including a calendar, a categories tree and an archive link list that are built based on index information of the content store. The rest of the page, header and footer elements aside, contains the articles, which are composed onto the page based on a set of rules and there's some content aggregation going on to produce, for instance, the comment counter. Each of these jobs takes a little time and they are worked on sequentially, while the data is acquired from the backend, the web-site (rendering) thread sits idle.
Likewise, imagine you have an intranet web portal that's customizable and gives the user all sorts of individual information like the items on his/her task list, the unread mails, today's weather at his/her location, a list of current projects and their status, etc. All of these are little visual components on the page that are individually requested and each data item takes some time (even if not much) to acquire. Likely more than here on this dasBlog site. And all the information comes from several, distributed services with the portal page providing the visual aggregation. Again, usually all these things are worked on sequentially. If you have a dozen of those elements on a page and it takes unusually long to get one of them, you'll still sit there and wait for the whole dozen. If the browser times out on you during the wait, you won't get anything, even if 11 out of 12 information items could be acquired.
One aspect of what I am working having all those 12 things done at the same time and allow the rendering thread to do a little bit of work whenever one of the items is ready and to allow the page to proceed whenever it loses its patience with one or more of those jobs. So all of the data acquisition work happens in parallel rather than in sequence and the results can be collected and processed in random order and as they are ready. What's really exciting about this from an SOA perspective is that I am killing request/response in the process. The model sits entirely on one-way messaging. No return values, not output parameters anywhere in sight.
In case you wondered why it is so silent around here ... that's why.
Daniel Fisher aka "Lenny Bacon" joined the pack and is already having some fun.
Microsoft Watch highlights the recently surfaced HP memo that speculates that Microsoft would start enforcing its patent portfolio on Open Source. How likely is it? It is an interesting question, indeed. Here’s what I think:
The patent situation, especially on the middleware market, used to be very much like the cold war between the USSR and the USA in the last century. One side moves, everyone dies. My guess is that if Microsoft had gone out and dragged Sun to court over J2EE and Sun had countersued over .NET, things would have gotten really, really nasty. The very foundations of the J2EE stack are sitting right in the middle of a substantial Microsoft patent minefield covering what we know as MTS and COM+. The reverse doesn’t look much better. Now Sun and Microsoft made peace on that front and are even looking to negotiate a broad IP cross-licensing deal to end that sort of arms race. Cross-licensing of patents is quite common in this industry and in most other industries as well. So where does that leave the grassroots Open Source movement? Not in a good place, for sure.
If you do research and you pour millions or even billions into that research, there has to be some return on that investment. And there is a difference between academic research and research that yields commercial products. I am not saying that there is no close relationship between the two, but both are done with different goals. If you do research for commercial purposes, regardless of whether you do it in the automotive industry or in the pharmaceutical industry or even in the software industry, the results of your research deserve protection. At the same time, it is harmful to the society at-large, if everyone would keep all results of all research under wraps. So governments offer companies a deal: You disclose the results of your research and we grant you a limited-time monopoly to use that technology exclusively. If you decide to share the technology with other parties, you can be forced to allow third parties to license it on appropriate terms. And the German Patent law §11 (1) , for instance, explicitly states that patents do not cover noncommercial use of technology in a private environment.
Now, if states offer that sort of system, a company that is built almost only on intellectual property (like Sun, IBM, Oracle, Apple, Microsoft and so forth) must play with the system. The must file for patents. If they don’t, they end of with something like the Eolas mess in their hands and that is not pretty. Even if some of the patents seem absolutely ridiculous; if the patent lawyers at a large company figure out that a certain technology is not covered by an existing patent, they must go and protect it. Not necessarily to enforce it, but rather to avoid that someone else enforces it on them. And because a lot of these patents are indeed idiotic, such are rarely ever enforced and most often quite liberally licensed. Something similar is true for trademarks. Microsoft has no choice but to chase Lindows (now Linspire) or even poor “Mike Rowe Soft” because they must defend their trademarks, by law. If they don’t and let a case slip, they might lose them. It’s not about being nasty, it’s just following the rules that lawmakers have set.
Now, if someone starts cheating on the research front and consumes IP from that system but never contributes IP to the system, it does indeed change the ballgame. If you don’t have a patent portfolio that is interesting enough for someone else to enter a (possibly scoped) cross-licensing deal with you and you don’t license such patents for money but instead break the other parties’ rightfully (it’s the law!) acquired, time-limited monopolies on commercial use of the respective technologies and you do so for profit, then you are simply violating the rules of the law. That’s as simple as it is. So, if I held Sun’s or Microsoft’s patent portfolio, would I ask those who profit from commercialization of those patents for my share? I really might give it some serious consideration. I think companies like Red Hat make wonderful targets, because they are commercial entities that profit greatly from a lot of IP that they do not (as I suspect) have properly licensed for commercial exploitation. The interesting this is that my reading of the (German) patent law is that the non-profit Apache Foundation can actually use patented technology without being at risk, but a for-profit company cannot adopt their results without being liable to acquire a license. Even giving away “free” software in order to benefit from the support services is commercialization. So if Red Hat includes some Apache project’s code that steps on patents, I’d say they are in trouble.
Now, if someone were to “reimplement” a patented drug, the pharmaceutical company sitting on the patent would sue them out of existence the next second without even blinking. Unless I am really badly informed, the entire biotech industry is entirely built on IP protection. All these small biotech firms are doing research that eventually yields protected IP and that’s what they look to turn into profit. They’re not in the business of producing and distributing the resulting drugs on a world-wide scale, they look to share the wealth with the pharmaceutical giants that have the respective infrastructure. The software industry is a very, very tame place against what’s going on in other industries. So will Sun, IBM, Oracle, Apple, and/or Microsoft eventually become more serious about drawing profit from the rights they hold? Right now it would be a very, very stupid thing to do in terms of the resulting, adverse marketing effect.
Now imagine Sun’s unfortunate decline keeps going or some other technology company with a substantial patent portfolio (and not some weak copyright claims) falls into the hands of a litigious bunch of folks as in the case of SCO. That’s when the shit is going to hit the fan. Big time.
Unless you enable the config setting below, WSE injects intentionally invalid “Via” routing information into ReplyTo and FaultTo addresses for security reasons and therefore you can’t just turn around and create, for instance, a new SoapSender(SoapRequestContext.Address.ReplyTo) at the receiving endpoint or set the reply envelope’s context like envelope.Context.Addressing.Destination = SoapRequestContext.Address.ReplyTo. Because “Via” trumps any other address in an endpoint reference for delivery, a reply to such an invalidated EPR will usually yield a 404. I fell into that hole for the second or third time now and it somehow never stuck in long-term memory, so this is the persisted “note to self” 
<microsoft.web.services2>
<messaging>
<allowRedirectedResponses enabled="true" />
</messaging>
</microsoft.web.services2>
I was a little off when I compared my problem here to a tail call. Gordon Weakliem corrected me with the term "continuation".
The fact that the post got 28 comments shows that this seems to be an interesting problem and, naming aside, it is indeed a tricky thing to implement in a framework when the programming language you use (C# in my case) doesn't support the construct. What's specifically tricky about the concrete case that I have is that I don't know where I am yielding control to at the time when I make the respective call.
I'll recap. Assume there is the following call
CustomerService cs = new CustomerService();
cs.FindCustomer(customerId);
FindCustomer is a call that will not return any result as a return value. Instead, the invoked service comes back into the caller's program at some completely different place such this:
[WebMethod]
public void FindCustomerReply(Customer[] result)
{
...
}
So what we have here is a "duplex" conversation. The result of an operation initiated by an outbound message (call) is received, some time later, through an inbound message (call), but not on the same thread and not on the same "object". You could say that this is a callback, but that's not precisely what it is, because a "callback" usually happens while the initiating call (as above FindCustomer) has not yet returned back to its scope or at least while the initiating object (or an object passed by some sort of reference) is still alive. Here, instead, processing of the FindCustomer call may take a while and the initiating thread and the initiating object may be long gone when the answer is ready.
Now, the additional issue I have is that at the time when the FindCustomer call is made, it is not known what "FindCustomerReply" message handler it going to be processing the result and it is really not know what's happening next. The decision about what happens next and which handler is chosen is dependent on several factors, including the time that it takes to receive the result. If the FindCustomer is called from a web-page and the service providing FindCustomer drops a result at the caller's doorstep within 2-3 seconds [1], the FindCustomerReply handler can go and hijack the initial call's thread (and HTTP context) and render a page showing the result. If the reply takes longer, the web-page (the caller) may lose its patience [2] and choose to continue by rendering a page that says "We are sending the result to your email account." and the message handler with not throw HTML into an HTTP response on an open socket, but rather render it to an email and send it via SMTP and maybe even alert the user through his/her Instant Messenger when/if the result arrives.
[1] HTTP Request => FindCustomer() =?> "FindCustomerReply" => yield to CustomerList.aspx => HTTP Response
[2] HTTP Request => FindCustomer() =?> Timeout! => yield to YouWillGetMail.aspx => HTTP Response
T+n =?> "FindCustomerReply" => SMTP Mail
=> IM Notification
So, in case [1] I need to correlate the reply with the request and continue processing on the original thread. In case [2], the original thread continues on a "default path" without an available reply and the reply is processed on (possibly two) independent threads and using two different notification channels.
A slightly different angle. Consider a workflow application environment in a bank, where users are assigned tasks and simply fetch the next thing from the to-do list (by clicking a link in an HTML-rendered list). The reply that results from "LookupAndDoNextTask" is a message that contains the job that the user is supposed to do.
[1] HTTP Request => LookupAndDoNextTask() =?> Job: "Call Customer" => yield to CallCustomer.aspx => HTTP Response
[2] HTTP Request => LookupAndDoNextTask() =?> Job: "Review Credit Offer" => yield to ReviewCredit.aspx => HTTP Response
[3] HTTP Request => LookupAndDoNextTask() =?> Job: "Approve Mortgage" => yield to ApproveMortgage.aspx => HTTP Response
[4] HTTP Request => LookupAndDoNextTask() =?> No Job / Timeout => yield to Solitaire.aspx => HTTP Response
In all of these cases, calls to "FindCustomer()" and "LookupAndDoTask()" that are made from the code that deals with the incoming request will (at least in the theoretical model) never return to their caller and the thread will continue to execute in a different context that is "TBD" at the time of the call. By the time the call stack is unwound and the initiating call (like FindCustomer) indeed returns, the request is therefore fully processed and the caller may not perform any further actions.
So the issue at hand is to make that fact clear in the programming model. In ASP.NET, there is a single construct called "Server.Transfer()" for that sort of continuation, but it's very specific to ASP.NET and requires that the caller knows where you want to yield control to. In the case I have here, the caller knows that it is surrendering the thread to some other handler, but it doesn't know to to whom, because this is dynamically determined by the underlying frameworks. All that's visible and should be visible in the code is a "normal" method call.
cs.FindCustomer(customerId) might therefore not be a good name, because it looks "too normal". And of course I don't have the powers to invent a new statement for the C# language like continue(cs.FindCustomer(customerId)) that would result in a continuation that simply doesn't return to the call location. Since I can't do that, there has to be a different way to flag it. Sure, I could put an attribute on the method, but Intellisense wouldn't show that, would it? So it seems the best way is to have a convention of prefixing the method name.
There were a bunch of ideas in the comments for method-name prefixes. Here is a selection:
- cs.InitiateFindCustomer(customerId)
- cs.YieldFindCustomer(customerId)
- cs.YieldToFindCustomer(customerId)
- cs.InjectFindCustomer(customerId)
- cs.PlaceRequestFindCustomer(customerId)
- cs.PostRequestFindCustomer(customerId)
I've got most of the underlying correlation and dispatch infrastructure sitting here, but finding a good programming model for that sort of behavior is quite difficult.
[Of course, this post won't make it on Microsoft Watch, eWeek or The Register]
News is what is made news.
Point in case: This sentence on my blog here: "There's apparently a related project Boa (another serpent name along the family line of Viper that was the original codename for MTS), including the business markup language BML (pronounced "Bimmel") that he's involved in and he talked a bit about that, but of course I'd be killed if I gave out more details." now prompts, directly or indirectly, this here on Microsoft Watch and this on eWeek.
Nobody said that the project was software in product development. Nobody said it was about stuff that would eventually ship. Nobody really said anything that would be in any way relevant to technical or business decision makers today. What this shows is that there's a bit too much appetite for the next big thing while we're all still working on making the current big thing happen. Do you seriously think I am someone who'd casually leak Microsoft trade secrets on his blog?
And.... seriously.... go back and read the first six sentences on that entry with your brain switched into "active mode".
The recording of last week's .NET Rocks show on which I explained my view on the "services mindset" (at 4AM in the morning) is now available for download from franklins.net
|