Having an Internet Service Bus up in the cloud is not very entertaining unless there are services in the bus. Therefore, I built one (and already showed some of the code basics) that’s hopefully fun to play with and will soon share the first version with you after some scrubbing and pending a few updates to the ISB that will optimize the authentication process. It’s a 0.1 version and an experiment. The code download should be ready in the next two weeks, including those adjustments. But you can actually play with parts of it today without compiling or installing anything. The info is at the bottom of this post.

To make matters really interesting, this sample not only shows how to plug a service into the cloud and call it from some Console app, but is a combo of two rather unusual hosts for WCF services: A Windows Live Messenger Add-In that acts as the server, and a Windows Vista Sidebar gadget that acts as the client.

Since the Silicon Valley scene is currently all over Twitter and clones of Twitter are apparently popping up somewhere every day, I thought I could easily provide fodder to the proponents of the alleged Microsoft tradition of purely relying on copying other’s ideas and clone them as well ;-)  Well, no, maybe not. This is a bit different.

TweetieBot is an example of a simple personal service. If you choose to host it, you own it, you run it, you control it. The data is held nowhere but on your personal machine and it’s using the BizTalk Services ISB to stick its head up into the cloud and at a stable endpoint so that its easily reachable for a circle of friends, bridging the common obstacles of dynamic IPs, firewalls and NAT. No need to use UPnP or open up ports on your router. If you choose to do so, you can encrypt traffic so that there’s no chance that anyone looking at our ISB nor anyone else can see the what’s actually going across the wire.

Right now, lots of the Web 2.0 world lives on the assumption that everything needs to live at central places and that community forms around ad-driven hubs. The mainframe folks had a similar stance in the 70s and 80s and then Personal Computers came along. The pendulum is always swinging and I have little doubt that it will swing back to “personal” once more and that the federation of personal services will seriously challenge the hub model once more.

So what does the sample do? As indicated, TweetieBot is a bot that plugs into a Windows Live Messenger using a simple Add-In. Bart De Smet has a brilliant summary for how to build such Add-Ins. When the Add-In is active and someone chats the bot, it answers politely and remembers the chat line, time and sender. The bird has a leaky long term memory, though. It forgets everything past the last 40 lines.

Where it gets interesting is that the Add-In can stick three endpoints into the BizTalk Services ISB:

  • A Request/Response Web Service that allows retrieving the list of the last 40 (or less) “tweets” and also allows client to submit tweets programmatically.
  • An RSS service that allows (right now) anyone to peek in to the chat log of the last 40 tweets.
  • An Event service that allows subscribers to get real-time notifications whenever a new tweet is recorded.

The accompanying Sidebar Gadget, which is implemented using WPF, is a client for two of these services.

 When you drop the Gadget on the Sidebar, it will prompt for the IM address of the TweetieBot service you’d like to subscribe to. Once you’ve authenticated at the relay using your registered Information Card, the gadget will pull and show the current list of Tweets and subscribe to the Events service for real-time updates. And whenever someone chats the bot, the Sidebar gadget will immediately show the new entry. So even though the Gadget lives on some client machine that’s hidden between several layers of firewalls and behind NAT, it can actually get push-style event notifications through the cloud!

“How do I send events to clients?” must be one of the most frequent questions that I’ve been asked about Web Services in the past several years. Well, this is your answer right here.

While I’m still toying around with the code and the guys on the 1st floor in my building are doing some tweaks on the ISB infrastructure to make multi-endpoint authentication simpler, you can already play with the bot and help me a bit:

Using Windows Live Messenger you can chat (click here) tweetiebot@hotmail.com now. Drop a few lines. If the bot is online (which means that I’m not tinkering with it) it will reply. Then look at this RSS feed [1] and you can see what you and everyone else have been telling the bot recently. Enjoy.

[1] http://connect.biztalk.net/services/tweetiebot/tweetiebot%40hotmail.com/rss

Categories: Technology | BizTalk | ISB | WCF

May 7, 2007
@ 11:00 PM

Steve Maine explains what's in the newest revision of the BizTalk Services SDK, including quite a few (standalone-) surprises for WCF and WF developers. In case you haven't noticed, we've dropped a new and substantially expanded build of the SDK just a week after we published the first SDK.

Stop. Don't leave yet. Before you say "What do I care about BizTalk?", you should know that while BizTalk has been more or less associated with the BizTalk Server 200x product line in the past few years, (Codename-) BizTalk Services is a complementary set of functionality that's not only interesting to BizTalk Server customers, but really to all .NET developers.

Weird? Flip flopping? Confusing? No. The fact that BizTalk is not only BizTalk Server isn't really new. When BizTalk came out back in 2000 and I was very closely looking at what's going on (get it used for $2), the definition read like this in the press release:

The BizTalk Initiative represents the collective set of investments that Microsoft is making to facilitate business process integration within and between organizations using Internet-standard protocols and formats. It includes the BizTalk Framework, the BizTalk.org community and business document library, as well as BizTalk Server 2000, a business process orchestration server and tools for developing, executing and managing distributed business processes. These investments are being made in conjunction with industry standards groups, technology and service providers, as well as key global organizations.

While the envisioned schema exchange BizTalk.org fell flat since industry-wide message-level-schema standardization for "everything" more or less didn't happen in the way people initially expected, what came out of this initiative as a significant element was that the set of specifications then known as the BizTalk Framework 2.0 that acted as a foundation for quite a few of the WS-* specifications and the BizTalk Server product which evolved into a very successful and leading SOA/BPM suite that's soon seeing its next release, BizTalk Server 2006 R2. Fast forward, read Steven Martin's blog entry where he writes:

[...] We see BizTalk Services as a complement to "traditional" BizTalk Server uses on premise. As you need to coordinate SOA on a broader scale beyond the organization, we see the introduction of hosted services as one way to help support federation of business process, messaging, and identity across boundaries. Over time, we want to ensure that BizTalk Server customers will be able to easily use the cloud services in conjunction with their premise technology. [...]

So all in all, a very sane way to think about BizTalk is that the software and services we publish under that name are providing functionality for messaging, process management and connectivity that go beyond the capability of the core .NET Framework.

Categories: BizTalk | CardSpace | ISB

I wrote a slightly Twitter-inspired, fun app over the weekend that's using the BizTalk Services Connectivity service and relay. In the spirit of Software+Services I'm going to give you half of it [for now] ;-)   You must have the BizTalk Services SDK installed to run the sample.

The server app, which I'm keeping to myself for the next few days as part of the experiment, is an extension (add-in) to Windows Live Messenger. The Messenger add-in monitors all chats with tweetiebot@hotmail.com and keeps circular buffer with the last 40 incoming messages. Using the client (which is in the attached archive), you can get a list of "Tweets" and add a new one (same as chatting)

[ServiceContract(Name = "TweetieBot", Namespace = http://samples.vasters.com/2007/05/tweetiebot)]
public interface ITweetieBot
{
  [OperationContract]
  IList<Tweet> GetTweets(DateTime? since);
  [OperationContract]
  void Tweet(string nickname, string text);
}

or you can subscribe to new tweets and get them as they arrive

[ServiceContract(Name = "TweetieEvents", Namespace = http://samples.vasters.com/2007/05/tweetiebot)]
public interface ITweetieEvents
{
  [OperationContract(IsOneWay=true)]
  void OnTweet(Tweet tweet);
}

The client application hooks up to the client (that lives right on my desktop machine) through the BizTalk Services ISB and the server fires events back through the ISB relay into the client as new tweets arrive. So when you run the attached client app, you'll find that it starts with a dump of the current log of the bot and then keeps spitting out events as they arrive.

The client is actually pretty simple. The EventsClient is the subscriber for the pub/sub service (ConnectionMode.RelayMulticast) that writes out the received events to the console. The rest all happens in Main (parsing an validating the command line argument) and in Run.

    class Program
    {
       
class EventsClient :
ITweetieEvents
        {
           
public void OnTweet(Tweet
tweet)
            {
               
Console.WriteLine("[{0}] {1}:{2}"
, tweet.Time, tweet.User, tweet.Text);
            }
        }

       
static void Main(string
[] args)
        {
           
string usageMessage = "Usage: IMBotClient <messenger-email-address>"
;
           
if
(args.Length == 0)
            {
               
Console
.WriteLine(usageMessage);
            }
           
else
            {
               
if (!Regex.IsMatch(args[0], @"^([\w\-\.]+)@((\[([0-9]{1,3}\.){3}[0-9]{1,3}\])|(([\w\-]+\.)+)([a-zA-Z]{2,4}))$"
))
                {
                   
Console
.WriteLine(usageMessage);
                   
Console.WriteLine("'{0}' is not a valid email address"
);
                }
               
else
                {
                    Run(args[0]);
                }
            }
        }

       
private static void Run(string
emailAddress)
        {
           
EndpointAddress serviceAddress =
               
new EndpointAddress(String.Format(String.Format("sb://{0}/services/tweetiebot/{1}/service"
                                    
RelayBinding.DefaultRelayHostName, Uri
.EscapeDataString(emailAddress))));
           
EndpointAddress eventsAddress =
               
new EndpointAddress(String.Format(String.Format("sb://{0}/services/tweetiebot/{1}/events",
                                   
RelayBinding.DefaultRelayHostName, Uri
.EscapeDataString(emailAddress))));

The URI scheme for services that hook into the ISB is "sb:" and the default address of the relay is encoded in the SDK assemblies. We set up two endpoints here. One for the client channel to fetch the initial list and one for the event subscriber. 

            RelayBinding relayBinding = new RelayBinding();
     

            
ServiceHost eventsHost = new ServiceHost(typeof(EventsClient
));
           
RelayBinding eventBinding = new RelayBinding(RelayConnectionMode
.RelayedMulticast);
            eventsHost.AddServiceEndpoint(
typeof(ITweetieEvents
), eventBinding, eventsAddress.ToString());
            eventsHost.Open();

           
ChannelFactory<TweetieBotChannel> channelFactory = new ChannelFactory<TweetieBotChannel
>(relayBinding, serviceAddress);
           
TweetieBotChannel
channel = channelFactory.CreateChannel();
            channel.Open();

The two *.Open() calls will each prompt for a CardSpace authentication, so you will have to be registered to run the sample. Once you have opened the channels (and my service is running), you'll be able to pull the list of current tweets. Meanwhile, whenever a new event pops up, the EventsClient above will write out a new line.

            IList<Tweet> tweets = channel.GetTweets(lastTime);
           
foreach (Tweet tweet in
tweets)
            {
               
Console.WriteLine("[{0}] {1}:{2}"
, tweet.Time, tweet.User, tweet.Text);
            }

           
Console.WriteLine("Press ENTER to quit at any time"
);
           
Console
.ReadLine();

            eventsHost.Close();
            channel.Close();
            channelFactory.Close();
        }        


So when you run the app, you can chat (anyone can, you don't need to be a buddy) tweetiebot@hotmail.com through Live Messenger and you'll see your chat lines (and potentially others') popping out as events from the service bus.

To run the sample with my bot, you need to call the client with "IMBotClient tweetiebot@hotmail.com" and select your BizTalk Services Information Card twice as you are prompted.

Privacy notice: I'm anonymizing the name of the contact only insofar as I'm clipping anything including and following the "at" sign of the user that chats the bot. So whatever you say is published as "emailname: text line"

IMBotClient.zip (3.61 KB)
Categories: Technology | BizTalk | CardSpace | ISB

Christian Weyer shows off the few lines of pretty straightforward WCF code & config he needed to figure out in order to set up a duplex conversation through BizTalk Services.

Categories: Architecture | SOA | BizTalk | WCF | Web Services | XML

Steve has a great analysis of what BizTalk Services means for Corzen and how he views it in the broader industry context.

Categories: Architecture | SOA | IT Strategy | Technology | BizTalk | WCF | Web Services

April 25, 2007
@ 03:28 AM

"ESB" (for "Enterprise Service Bus") is an acronym floating around in the SOA/BPM space for quite a while now. The notion is that you have a set of shared services in an enterprise that act as a shared foundation for discovering, connecting and federating services. That's a good thing and there's not much of a debate about the usefulness, except whether ESB is the actual term is being used to describe this service fabric or whether there's a concrete product with that name. Microsoft has, for instance, directory services, the UDDI registry, and our P2P resolution services that contribute to the discovery portion, we've got BizTalk Server as a scalable business process, integration and federation hub, we've got the Windows Communication Foundation for building service oriented applications and endpoints, we've got the Windows Workflow Foundation for building workflow-driven endpoint applications, and we have the Identity Platform with ILM/MIIS, ADFS, and CardSpace that provides the federated identity backplane.

Today, the division I work in (Connected Systems Division) has announced BizTalk Services, which John Shewchuk explains here and Dennis Pilarinos drills into here.

Two aspects that make the idea of a "service bus" generally very attractive are that the service bus enables identity federation and connectivity federation. This idea gets far more interesting and more broadly applicable when we remove the "Enterprise" constraint from ESB it and put "Internet" into its place, thus elevating it to an "Internet Services Bus", or ISB. If we look at the most popular Internet-dependent applications outside of the browser these days, like the many Instant Messaging apps, BitTorrent, Limewire, VoIP, Orb/Slingbox, Skype, Halo, Project Gotham Racing, and others, many of them depend on one or two key services must be provided for each of them: Identity Federation (or, in absence of that, a central identity service) and some sort of message relay in order to connect up two or more application instances that each sit behind firewalls - and at the very least some stable, shared rendezvous point or directory to seed P2P connections. The question "how does Messenger work?" has, from an high-level architecture perspective a simple answer: The Messenger "switchboard" acts as a message relay.

The problem gets really juicy when we look at the reality of what connecting such applications means and what an ISV (or you!) were to come up with the next cool thing on the Internet:

You'll soon find out that you will have to run a whole lot of server infrastructure and the routing of all of that traffic goes through your pipes. If your cool thing involves moving lots of large files around (let's say you'd want to build a photo sharing app like the very unfortunately deceased Microsoft Max) you'd suddenly find yourself running some significant sets of pipes (tubes?) into your basement even though your users are just passing data from one place to the next. That's a killer for lots of good ideas as this represents a significant entry barrier. Interesting stuff can get popular very, very fast these days and sometimes faster than you can say "Venture Capital".

Messenger runs such infrastructure. And the need for such infrastructure was indeed an (not entirely unexpected) important takeaway from the cited Max project. What looked just to be a very polished and cool client app to showcase all the Vista and NETFX 3.0 goodness was just the tip of a significant iceberg of (just as cool) server functionality that was running in a Microsoft data center to make the sharing experience as seamless and easy as it was. Once you want to do cool stuff that goes beyond the request/response browser thing, you easily end up running a data center. And people will quickly think that your application sucks if that data center doesn't "just work". And that translates into several "nines" in terms of availability in my book. And that'll cost you.

As cool as Flickr and YouTube are, I don't think of none of them or their brethren to be nearly as disruptive in terms of architectural paradigm shift and long-term technology impact as Napster, ICQ and Skype were as they appeared on the scene. YouTube is just a place with interesting content. ICQ changed the world of collaboration. Napster's and Skype's impact changed and is changing entire industries. The Internet is far more and has more potential than just having some shared, mashed-up places where lots of people go to consume, search and upload stuff. "Personal computing" where I'm in control of MY stuff and share between MY places from wherever I happen to be and NOT giving that data to someone else so that they can decorate my stuff with ads has a future. The pendulum will swing back. I want to be able to take a family picture with my digital camera and snap that into a digital picture frame at my dad's house at the push of a button without some "place" being in the middle of that. The picture frame just has to be able to stick its head out to a place where my camera can talk to it so that it can accept that picture and know that it's me who is sending it.

Another personal, and very concrete and real point in case: I am running, and I've written about that before, a custom-built (software/hardware) combo of two machines (one in Germany, one here in the US) that provide me and my family with full Windows Media Center embedded access to live and recorded TV along with electronic program guide data for 45+ German TV channels, Sports Pay-TV included. The work of getting the connectivity right (dynamic DNS, port mappings, firewall holes), dealing with the bandwidth constraints and shielding this against unwanted access were ridiculously complicated. This solution and IP telephony and video conferencing (over Messenger, Skype) are shrinking the distance to home to what's effectively just the inconvenience of the time difference of 9 hours and that we don't see family and friends in person all that often. Otherwise we're completely "plugged in" on what's going on at home and in Germany in general. That's an immediate and huge improvement of the quality of living for us, is enabled by the Internet, and has very little to do with "the Web", let alone "Web 2.0" - except that my Program Guide app for Media Center happens to be an AJAX app today. Using BizTalk Services would throw out a whole lot of complexity that I had to deal with myself, especially on the access control/identity and connectivity and discoverability fronts. Of course, as I've done it the hard way and it's working to a degree that my wife is very happy with it as it stands (which is the customer satisfaction metric that matters here), I'm not making changes for technology's sake until I'm attacking the next revision of this or I'll wait for one of the alternative and improving solutions (Orb is on a good path) to catch up with what I have.

But I digress. Just as much as the services that were just announced (and the ones that are lined up to follow) are a potential enabler for new Napster/ICQ/Skype type consumer space applications from innovative companies who don't have the capacity or expertise to run their own data center, they are also and just as importantly the "Small and Medium Enterprise Service Bus".

If you are an ISV catering shrink-wrapped business solutions to SMEs whose network infrastructure may be as simple as a DSL line (with dynamic IP) that goes into a (wireless) hub and is as locked down as it possibly can be by the local networking company that services them, we can do as much as we want as an industry in trying to make inter-company B2B work and expand it to SMEs; your customers just aren't playing in that game if they can't get over these basic connectivity hurdles.

Your app, that lives behind the firewall shield and NAT and a dynamic IP, doesn't have a stable, public place where it can publish its endpoints and you have no way to federate identity (and access control) unless you are doing some pretty invasive surgery on their network setup or you end up building and running run a bunch of infrastructure on-site or for them. And that's the same problem as the mentioned consumer apps have. Even more so, if you look at the list of "coming soon" services, you'll find that problems like relaying events or coordinating work with workflows are very suitable for many common use-cases in SME business applications once you imagine expanding their scope to inter-company collaboration.

So where's "Megacorp Enterprises" in that play? First of all, Megacorp isn't an island. Every Megacorp depends on lots of SME suppliers and retailers (or their equivalents in the respective lingo of the verticals). Plugging all of them directly into Megacorp's "ESB" often isn't feasible for lots of reasons and increasingly less so if the SME had a second or third (imagine that!) customer and/or supplier. 

Second, Megacorp isn't a uniform big entity. The count of "enterprise applications" running inside of Megacorp is measured in thousands rather than dozens. We're often inclined to think of SAP or Siebel when we think of enterprise applications, but the vast majority are much simpler and more scoped than that. It's not entirely ridiculous to think that some of those applications runs (gasp!) under someone's desk or in a cabinet in an extra room of a department. And it's also not entirely ridiculous to think that these applications are so vertical and special that their integration into the "ESB" gets continuously overridden by someone else's higher priorities and yet, the respective business department needs a very practical way to connect with partners now and be "connectable" even though it sits deeply inside the network thicket of Megacorp. While it is likely on every CIO's goal sheet to contain that sort of IT anarchy, it's a reality that needs answers in order to keep the business bring in the money.

Third, Megacorp needs to work with Gigacorp. To make it interesting, let's assume that Megacorp and Gigacorp don't like each other much and trust each other even less. They even compete. Yet, they've got to work on a standard and hence they need to collaborate. It turns out that this scenario is almost entirely the same as the "Panic! Our departments take IT in their own hands!" scenario described above. At most, Megacorp wants to give Gigacorp a rendezvous and identity federation point on neutral ground. So instead of letting Gigacorp on their ESB, they both hook their apps and their identity infrastructures into the ISB and let the ISB be the mediator in that play.

Bottom line: There are very many solution scenarios, of which I mentioned just a few, where "I" is a much more suitable scope than "E". Sometimes the appropriate scope is just "I", sometimes the appropriate scope is just "E". They key to achieve the agility that SOA strategies commonly promise is the ability to do the "E to I" scale-up whenever you need it in order to enable broader communication. If you need to elevate one or a set services from your ESB to Internet scope, you have the option to go and do so as appropriate and integrated with your identity infrastructure. And since this all strictly WS-* standards based, your "E" might actually be "whatever you happen to run today". BizTalk Services is the "I".

Or, in other words, this is a pretty big deal.

Categories: Architecture | SOA | IT Strategy | Microsoft | MSDN | BizTalk | WCF | Web Services

The request below has been handed to me by the BizTalk team here at Microsoft. If you have programmed in WCF, happen to be at or around the Microsoft Redmond campus at that time and want to help out, send an email until this Friday to uccoord at microsoft.com with the subject line "BizTalk Usability Study" to sign up:

 

 

Microsoft is conducting research on BizTalk Server, and are seeking Developers who have a working knowledge of this product and WCF.  If you are a current BizTalk Developer, with WCF experience, the team would like to invite you to participate in this research. 

 

Studies are currently being scheduled for Monday Oct 30 through Friday Nov 3, 2006 in Redmond, WA.  Each study will be scheduled at your convenience and will run approximately 2 hours.  This is a unique opportunity to provide feedback on the adapter creation process in BizTalk.  

 

Your input and participation is extremely valuable that helps ensure that your needs are met when interacting with BizTalk and WCF. If scheduled for a usability study, you will receive a retail software product selection for your time and feedback.  Some of the items include Office Pro and VisualStudio.NET

 

Categories: BizTalk | WCF