Three related links.

The next installment of my little blog-series "Why you want to use Enterprise Services for your .NET application" has the clumsy working title "Object services and management of expensive and limited resources". I just need to make the time to finish it; hopefully that works today or tomorrow. The first two parts are also in my stories list.

Once System.Diagnostics.StackWalk and "__asm mov [__EBP], ebp;" become useful tools, you realize that you've exhausted an infrastructure's extensibility hooks. Sigh!

COM was love

This isn't an Enterprise Services crusade. There's a lot of very useful stuff and I am pointing it out. That's all ;)

However, I think I should admit that I did indeed have a very long affair with the entire ES/COM+/MTS/COM family. {Is that a nasty thing to admit?}. The love went away, now we're just good friends.  

October 1992. The first time COM got into customer's hands. I got this CD and I was like "man, that's nuts! Way too complicated!". The first shipment came with docs that dealt in very deep and great detail with IUnknown, structured storage and especially IMoniker. I didn't understand the moniker stuff until two years later. What I understood was AddRef()/Release() and QueryInterface(). An implementation of IUnknown became my default-base class for anything in November 1992. Only about 3 months later I understood how "outer IUnknowns" and aggregation worked. A lot of code-rewriting followed ;). It took about a year to realize that not *everything* should be a COM class. And so it went on and on....

And I just realize... there was no 10th birthday party;) 

My friend Morten Abrahamsen from Norway, an exceptionally smart software architect with a very practical view on things and a lot of experience building large transactional systems, has written a comment very worth reading here in the blog that deserves to seen not only by the folks who accidentially click the comments link. Morten says: "Interesting how the struggle for singular performance clouds the overview of a flexible architecture that is resilient to change, versioning and implementation details.".

LRPC.

Shameless sales pitch: Little known, 18 month old book with an unfitting title about an interesting product that obviously doesn't really sell well. I am considering to go back to that material and do a rewrite which is going to be less (or not at all) product focused and instead centers around service-oriented architectures and document-driven designs. Chapters 1 & 2 can be read online at amazon.

Hey, Greg, what are you doing with that method signature [1] when you're porting that app to another data provider? What if a future revision of that method wants to add that customer to the database asynchronously and does so by stuffing it into a transactional queue, first?

[1] public int AddCustomer(CustInfo custInfo, SqlTransaction trans)

Exactly. You don't need any of COM+ (which is really what to call it) unless you are trying to use the DTC. The rest is all done by the runtime already as you point out. [Sam Gentile's Radio Weblog]

No. Show me a COM+ feature except the non-transactional incarnation of the shared property manager (AppDomain statics may do that), which is exactly or even similarly replicated in the runtime. App-specific roles aren't, object pooling isn't, JIT activation isn't, QC aren't, process model (thread pool, app pool, pause, disable, recycling) isn't, event monitoring isn't, LCE aren't, memory gates aren't, secure RPC isn't, security context isn't and this list isn't complete. It's simply not right, Sam. 

John Lam shares his thoughts on Enterprise Services and says that many features of ES are replicated throughout the framework and therefore he sees little need to use ES anymore except for distributed transactions. This was my exact position about a year ago. At the time, I posted a lengthy statement with almost the same arguments to the Microsoft Regional Regional Directors (non-public) mailing list and since then I had a lot of very valuable discussions with a lot of very smart folks who mapped out the differences between what's in the framework and what's in ES and who have helped me to understand that I simply wasn't right with what I was saying.

Let me go over what John highlights:

• Load balancing. Load balancing is a filter for work where the amount of work is either not predictable or can't be handled by a single system. You load balance as close to the "topmost" client as you can, handle parts of the load and reduce the need for load balancing downstream. If you have a website or web-service, you load balance the web-tier. Will you load balance the business logic tier? Possibly, but the load generated by one web-server towards its backend is typically sufficiently predictable to eliminate the need for component load balancing and rather make not-so-dynamic assignments of backend servers to a group of web servers. Component load balancing is only really good for when you can't load-balance the presentation tier (for instance if you have GUI clients) or if you have a huge spread in execution times for a single class (if you allow users to execute ad-hoc database queries).
• ASP.NET as an application host. Hosting your business logic there is good, if you are all stateless. If you need to keep and share expensive-to-acquire application state around (such as large caches) or if you need to guard a set of limited resources that are limited for your entire web-farm, hosting there has limitations. The most important limitation of hosting business logic there is security. In ASP.NET, everything happens within the security context of the external caller (or its delegate ASPNET) and that's a problematic thing. You will want to do certain things with elevated privileges in the context of a service account and LogonUser() isn't really what you want to do in that case.
• Roles. You can make your own user and role types in the .NET framework, but not many people do. You'd have to write your own admin tools, your own infrastructure and you'd have to provide a mapping to OS roles and users for infrastructure access. If you stick with OS roles (SAM or Active Directory groups, in essence) and use the PrincipalPermissionAttribute as a replacement for ES role-based security, you will lose a level of indirection. Instead of defining a role required to access a single method on a single class in a single application right there in that application, you will have to define that in Active Directory and have it replicate throughout your AD structure. There can be very many such roles.
• Object Pool: Object pooling is a good workaround to overcome limitations of OLE DB, but that's just one aspect. It's a generic semaphore for classes. It helps implementing write access to any resource with limited or no concurrency control (the art of handling FileStream.Lock() is often long forgotten), it'll help you pre-initialize and control access to things like 3270 terminal screen-scrapers with very limited permitted concurrent sessions or maybe interfaces to physical devices of which you only have one or four (like a metal-sheet press). Nothing you couldn't do without ES, but ... it's already there and the number of folks who don't want to spend the time implementing all the required infrastructure goo is substantial.
• Transactions: Even with a single resource manager, using ES for transactions is not a bad idea for complex systems. For the simplest case that a component method creates a transaction, does work and commits the transaction, using native database connections are a good thing and the fastest choice. If a component calls another component (which may call other components itself) and the transaction shall span those components, transaction management can easily get out of hand. You will have to pass the database connection and the transaction objects around (in the case of ADO.NET), you will have to negotiate who may commit or abort and you may have to collect votes on the outcome.  Also, since components are black-boxes, you will not always know whether a component you want to roll into your transaction doesn't require a second resource manager -- in which case your native database transaction can't be used. (Update: check out Ingo Rammer's comments on the same topic)

Still, I consider John's remarks as very valid for a large number of web-applications. COM+ no longer is the automatic default for hosting business logic as it was when stuff was implemented in ASP and VB6 components instead of ASP.NET. However, ASP.NET hasn't become the automatic default for hosting all things related to a web application, either. Your mileage may vary.

 

Why you want to use Enterprise Services for your .NET application
Part 1: Introduction
Part 2: Basic Architectural Considerations and the benefits of Processes and Process Models

UI -> BusinessLogic -> DataAccess. This three layer model is the most common way to separate functional blocks of applications. It's clear, simple and very obvious. If you're writing a web application, all stuff on top of ASP, JSP or ASP.NET is your UI layer, whatever is called by that layer is your business logic layer and that, in itself is split into a "logic" and into a "data access" layer, which serves to make your data access code more resilient against changes in data access technology, database product choice or, in the simplest and most common case, schema changes in the underlying data store. If you are writing a GUI application, all GUI-related functionality is in the top-layer and you may be able to use the same business logic layer as you are using for a web application. Ideally, all user-interface type and technology specific aspects are handled in the UI layer, and business logic shall be as resilient against changes in the UI as it is against changes in the data layer. That's why we have those layers.

I call them "layers", not "tiers". In my world (not necessarily in everyone else's), a layer is a purely logical concept. Layering is about separating functionally different areas of code. To me, a tier is a solely a physical concept. It is about how code gets distributed in a runtime environment. A GUI application, which ends up being compiled into a single EXE may be built using multiple separate layers just as much as a server application, which potentially maps each layer on a distinct physical tier. More likely is, though, that stuff from two or more layers gets mapped into a tier or that two tiers handle one layer. Example: SQL Server stored procedures and components manipulating ADO.NET DataSets using ad-hoc SQL are all things that belong to an data access layer. Still they are physically deployed in two places: Inside SQL Server and in a process that is accessing SQL Server from the outside. I would call that two separate tiers, but one layer. A "single EXE" GUI application has many layers, but possibly only one tier (if it does, for instance use the JetDB engine mapped into its own process space).

That's all well-known and very obvious to mostly everybody as long as the "UI" role is very obvious (Web front or GUI) and the "Data Store" role is very obvious (some RDBMS). Most commonly, (sub-)systems which feature this type of layers are running in "reactionary mode". They are triggered by some user activity and run one or more (potentially parallel) sequences of activities in response to the user activity. Not seldomly, architectural confusion begins whenever a system shall perform autonomous actions (for instance based on timers) or when the trigger for an activity is not a user, but some other binary lifeform. Where does that fit into the layering picture? Does it fit at all? Also, if my business code needs to invoke a remote system through a web service or needs to submit a document to a remote site using an infrastructure like BizTalk or if it simply wants to send an email via SMTP. Where does that go?

In my world, the acronym "UI" doesn't mean "user-interface", it means "use-case-interface". Everything that triggers any activity in the business logic layer is a "UI". A Web Services is a UI, a BizTalk Server application integration component (AIC) is a UI, a Windows Service process is a UI. The business code doesn't really care whether the current method call was originally triggered by a human being clicking anywhere on a remote screen. All such UIs can share a lot of code. Indeed, "business logic" is all code that is UI-agnostic in this expanded scenario.

The "Data Access" layer isn't just for databases. All code that accesses any functionality outside your own application and which is triggered by activities that root in your own application belongs there. If you call a remote web service or a remote application that's not under your own immediate control, you need to make your business code resilient against changes in those external applications. If you send a Word document attached to an Email via SMTP now, you may want to send an PDF document via other ways tomorrow. The fact that the information must be sent doesn't change, formatting and ways of sending does. So, I like to speak of "Infrastructure Access Layer" rather than "Data Access" layer to limit confusion.

What we're getting out of this are three separate layers of code. "Use-Case Interface", "Business Logic" and "Infrastructure Access". It's a good way to organize interfaces and code and works very well for working in large teams. What we're not getting out of this a consistent and reliable mapping to a runtime environment. Business logic will execute in the process space of  ASP.NET, a BizTalk AIC, a Windows Service or a desktop GUI application. All these process spaces are very different. The GUI EXE runs reliably as long as the user Alt-F4's it or until the machine becomes unresponsive courtesy of Windows Exploder. BizTalk will load and unload an AIC (and hence your entire layered model) for the duration of a single action. ASP.NET will load your code, but it'll sometimes recycle the process "suddenly" for various good reasons. A Windows service has a very predicable execution profile (starts/stops at boot/shutdown), but by itself it doesn't have a concept of communication with the outside world -- you'll have to make it an RPC or Remoting server or a Message Queue listener yourself and that will involve creating and maintaining worker threads, etc.

If you want to write applications that are dealing with data efficiently and truly scale, you will want to cache large parts of those 80% of all tables in your data model that hold static or near-static data in memory. You will want to keep pools of infrastructure objects ready and initialized. You will want to have pre-activated and smart  "gatekeepers" that guard access to limited or expensive external resources such as 3270 terminal sessions, remote web services with low bandwidth, etc. What you need is a predictable execution environment, which allows you coordination of such access to limited resources, which will allow you to keep caches alive and current and which provides you with a security boundary that will allow authorization for accessing services and security-identity switches that can access services with elevated privileges for such authorized users. What you want is to go "out of process".

"Going out of process" and hosting your business logic in a decicated environment is not "necessary evil", it's a carefully chosen and intended feature of your architecture. Enterprise Services/COM+ (and J2EE application servers) provide you with such a predictable hosting environment for your "Business Logic" and "Infrastructure Layer" components. "Going out of process" means that you will isolate your business code from unique behavior of your "Use-Case Interface" hosting environment.

What you get is a process with a well-defined process model. It'll create and manage thread pools, it'll manage external access, it'll provide you with a way to access this functionality from other processes. Using Enterprise Services applications "in process" is a special case for whenever you only deal with a single "Use-case Interface" and you are ready to deal with possible restrictions its process model imposes for your business logic. Hosting business logic "out of process" is the default.

That's why you want an application server environment. Enterprise Services/COM+ and most J2EE application servers provide such an environment. These principles count on the server, but also on the desktop. Paying the price of cross-process marshaling is not something you are forced to do under torture, it's something which you'll do because you'll get something for that price. Power = Work /Time.

Today's favorite Enterprise Services attribute: [assembly:ApplicationActivation(ActivationOption.Server)]

Next installment: Part 3: Management of expensive and limited resources

 

Why you want to use Enterprise Services for your .NET application
Part 1: Introduction

Yesterday I did a 4.5 hour talk about the relevance and basics of Enterprise Services here at TornadoCamp.NET. In our audience we have about 90% developers who have been using mostly VB6/VB5 up until now and more than half are writing "classic" client/server applications with (very) fat clients and the only server-side actions happening inside SQL Server, Oracle, Sybase by way of stored procedures. What I've found here is consistent with what I find at our other workshops and very many other events where I speak: Only very few developers really ever used COM+ or MTS for anything but server-side transaction handling and the majority didn't even look at Enterprise Services/COM+/MTS ever, at all.

Why that is the case is easily explained and there are two primary reasons:

(a) Visual Basic 6 (and previous versions) is the most popular language for writing business applications on Windows, at least with our customers and the people I usually talk to at conferences and events. COM+ provides quite a few very useful features, which either can't be used from within the VB's "STA ghetto" due to its inability to produce thread-safe code (like ObjectPooling) or which are very difficult to deploy without rather complex installation scripts (like "loosely coupled events") .

(b) The main reason is a different one: COM+ provides the implementation of a lot of common architectural patterns and solutions to very typical functional challenges. If I either don't understand these patterns or, more often, don't see an obvious mapping of such a functional challenge that I find in my project to a feature provided by COM+, I simply won't use it. The dilemma: If you don't really know what's in COM+ feature bag, you won't be able to find out why you'd ever want to consider using it. If you have no interest in COM+, you will not buy a book on it. For most developers, all feature areas of COM+ beyond "Transaction.Required" therefore remain in the dark.

So, instead of blogging random Enterprise Services features out-of-context (such as CoRegisterSurrogateEx), I will try to illustrate the "why" and use-cases for several (best:all)  Enterprise Services/COM+ services in a very compact, blog compatible form, which will hopefully create a context for the other obscure things I am typically writing about and will allow more people to see why this stuff is very relevant for their apps.

Next installment: Part 2: Basic Architectural Considerations and the benefits of Processes and Process Models

 

I start to wonder whether it may make sense to do a conference or tour really dedicated to Enterprise Services in XP/.NET Server.

CoRegisterSurrogateEx continued: Tomas comments here in the blog: "Let me see if I get one thing straight: You say we should _enable_ the application when the host starts up and _disable_ it when it shuts down, right?"   --- Yes, exactly.

"While we're on it, I think in many cases it would just be enough (and easier, up to a point), to simply make the application components fail activation if they're not running inside your custom surrogate.... what do you think?" -- That would be a good safeguard to avoid failures causes by an incorrect "activation environment". Still, throwing exceptions at activation time is a good indicator that something is wrong, but it won't help you spinning up a working host. Also, once a "wong host" is running, you'll have problems getting the custom host to run properly. Therefore, disabling or "run as NT services" is the better idea

Summary: You can host COM+ server applications in a custom host process, with all features. In fact, the host process doesn't need to be fully dedicated to hosting the COM+ app. You just need to spin up one thread for CoRegisterSurrogateEx (which in turn will spawn all thread required for the COM+ app), while other threads can do other things.  

Custom Surrogate. Well, the use of CoRegisterSurrogateEx() turned out to be easier than I expected. I buildt a small managed sample to... [Commonality]

A few comments on using CoRegisterSurrogateEx.

CoRegisterSurrogateEx expects the application id of an Enterprise Services/COM+ package. It "consumes" the calling thread and sets up the COM+ thread pools, etc. on top of the calling host.

Using CoRegisterSurrogateEx causes your server apps to behave the exact same way as if they were hosted by dllhost.exe. Dllhost.exe is a very thin wrapper around this exact API. If you launching the app from within managed code (as Tomas does it), the activated serviced components will be activated in the default domain of the host and free them from the "GAC ghetto".

Bad news: COM+ will ALWAYS start packages on top of dllhost.exe if an activation occurs and the package isn't running. That means that your custom host must be running before any activation takes place. The best ways to guarantee this are only available on COM+ 1.5:

(a) Disable/Enable the application when your own host spins up and shuts down. That requires that the custom host runs in the context of a principal with write access to the COM+ catalog.

(b) Host the application inside a managed Windows Service, register the COM+ app to "run as NT service" and patch the registry (yes, you heard right) for that service so that the service host is your own host and not dllhost.exe. The registry value to patch is under "SYSTEM\\CurrentControlSet\\Services\\{ServiceName}" and there the ImagePath" value. That must be set to your exe instead of dllhost.exe. The service must be set up to match the logon identity of the COM+ app. It can't be "interactive user", obviously.

If you grab my esutilities and try the EnterpriseServicesApplicationInstaller setting its "UseCustomSurrogateService" property to true, it'll dump a service process exe right next to the registered assembly and do the necessary registry patches. (Source code for some of these things isn't available but will be available in the forseeable future.)

On Windows 2000, you will probably have to register/unregister the whole COM+ app at startup/spindown as Disable/Enable isn't available there. Due to the auto-registration features of Enterprise Services that isn't really as bad as it sounds -- it's still bad, but not SO bad.