A service is said to be stateless if the consumer of that service can make use of any operating instance of that service. This is achieved by services not storing any internal data (state) that would be required if the consumer happened to invoke another instance of that service.
In general this goal is achieve by ensuring that all service data (including state) is kept in an external store common to all instances of that service.
However for performance reasons, service implementers are often inclined to cache back-end system query results or consumer session data. The reason for this is that the back-end systems and data stores which are accessed by services very often become points of contention and a source of bottlenecks, so avoiding the need to query those systems through caching at the service tier is attractive.
Service tier caching requires that cached data is available at all service instances, and that these instances are kept in sync both with each other and the underlying data store. Achieving this requires fairly sophisticated cache management software with cache refresh, propagation and invalidation mechanisms, as well as volatility analysis of the data being cached, i.e. the less volatile data is the more suitable it will be for caching.
JEE applications servers such as Weblogic offer HTTP and Stateful EJB session state caching that can be replicated across a cluster and persisted to a variety of data stores for failover purposes. In addition consumer connections to servers can be made “sticky” so that a given consumer will always return to the same server and only be redirected to another server in a failover scenario. This is fine as long as either the state is replicated or the consumer is happy to lose some conversational state in the rare instance that a server becomes unavailable. It is also possible to store consumer state on the client, either via client cookies or via URL rewriting. Weblogic for example can be configure to use either depending on whether client browser cookies are enable or not.
A viable and arguable simpler alternative to all of the above (where network latency between service and data tiers is not an issue) is to perform caching in the data tier using for example a DBMS feature such as Oracle RAC which provides a cluster of processes with data caches to improve performance and resilience.
Statelessness is a key characteristic of services which is essential for a scalable and fault tolerant SOA. State may be introduced if required but only after careful consideration as it will require complex and potentially less fault tolerant cache management facilities.