Stateless?!

Welcome back ;-)

On one of those flights last week I read a short article about Enterprise Services in a developer magazine (name withheld to protect the innocent). The “teaser” introduction of the article said: “Enterprise Services serviced components are scalable, because they are stateless.” That statement is of course an echo of the same claim found in many other articles about the topic and also in many write-ups about Web services. So, is it true? Unfortunately, it’s not.

public class AmIStateless {        public int MyMethod()        {               // do some work               return 0;        } }

 “Stateless” in the sense that it is being used in that article and many others describes the static structure of a class. Unfortunately that does not help us much to figure out how well instances of that class help us to scale by limiting the amount of server resources they consume. More precisely: If you look at a component and find that it doesn’t have any fields to hold data across calls (see the code snippet) and does furthermore not hold any data across calls in some secondary store (such as a “session object”), the component can be thought of as being stateless with regards to its callers, but how is the relationship with components and services that are called from it?

But before I continue: Why do we say that “stateless” scales well?

A component (or service) that does not hold any state across invocations has many benefits with regards to scalability. First, it lends itself well to load balancing. If you run the same component/service on a cluster of several machines and the client needs to make several calls, the client can walk up to any of the cluster machines in any order. That way, you can add any number of machines to the cluster and scale linearly. Second, components that don’t hold state across invocations can be discarded by the server at will or can be pooled and reused for any other client. This saves activation (construction) cost if you choose to pool and limits the amount of resources (memory, etc.) that instances consume on the server-end if you choose to discard components after each call. Pooling saves CPU cycles. Discarding saves memory and other resources. Both choices allow the server to serve more clients.

However, looking at the “edge” of a service isn’t enough and that’s where the problem lies.

The AmIStateless service that I am illustrating here does not stand alone. And even though it doesn’t keep any instance state in fields as you can see from the code snippet, it is absolutely not stateless. In fact, it may be a horrible resource hog. When the client makes a call to a method of the service (or otherwise sends a message to it), the service does its work by employing the components X and Y. Y in turn delegates work to an external service named ILiveElsewhere. All of a sudden, the oh-so-stateless AmIStateless service might turn into a significant resource consumer and limit scalability.

First observation: While no state is held in fields, the service does hold state on the stack while it runs. All local variables that are kept in on the call stack in the invoked service method, in X and in Y are consuming resources and depending on what you do that may not be little. Also, that memory will remain consumed until the next garbage collector run.

Second observation: If any of the secondary components takes a long time for processing (especially ILiveElsewhere), the service consumes and blocks a thread for a long time. Depending on how you invoke ILiveElsewhere you might indeed consume more than just the thread you run on.

Third observation:  If AmIStateless is the root of a transaction, you consume significant resources (locks) in all backend resource managers until the transaction completes – which may be much later than when the call returns. If you happen to run into an unfortunate situation, the transaction may take a significant time (minutes) to resolve.

Conclusion:  Since the whole purpose of what we usually do is data processing and we need to pass that data on between components, nothing is ever stateless while it runs. “Stateless” is a purely static view on code and only describes the immediate relationship between one provider and one consumer with regards to how much information is kept across calls. “Stateless” says nothing about what happens during a call.

Consequence: The scalability recipe isn’t to try achieving static statelessness by avoiding holding state across calls. Using this as a pattern certainly helps the naïve, but the actual goal is rather to keep sessions (interaction sequence duration) as short as possible and therefore limit the resource consumption of a single activity. A component that holds state across calls but for which the call sequence takes only a very short time or which does not block a lot of resources during the sequence may turn out to aid scalability much more than a component that seems “stateless” when you look at it, but which takes a long time for processing or consumes a lot of resources while processing the call. One way to get there is to avoid accumulating state on call stacks. How? Stay tuned.