BOF-2180: Effective Web Tier Clustering

Jules Gosnell (jules@coredevelopers.net)

Agenda

A brief look at the goals and problems of clustering webapps.

High Availability

Quality of Service

Managing State

What solutions are commonly applied to such problems ?

Pros & Cons

How will Apache Geronimo resolve these issues ?

WADI

Summary

Demonstration

Gotchas

Questions & Answers

Clustering: Goals

High Availability (HA)

Failover

increases availability exponentially

Quality of Service

Load Balancing

potential linear QoS improvements through horizontal scaling

Modern Load Balancers provide both fail-over and balancing functionality across a set of servers.

Clustering: Problem - State

In order for a user request entering a cluster to receive a correct response, its associated session must be available to the web container instance in which it is processed.

BROKEN:

Client

Server2

Server1

LoadBalancer

xxx

xxx

Stateful Clustering: Solution - Affinity

We add logic to the load-balancer so that it will always route stateful requests to the node holding the relevant session.

FIXED:

Client

Server2

Server1

LoadBalancer

xxx

xxx

Stateful Clustering - Affinity

Pros:

Simple: a little more complexity in load-balancer.

Cheap: a few more cycles expended in load-balancer

No changes needed to webapp or webcontainer.

Cons:

We are completely reliant on affinity.

No Fail-Over if node becomes unavailable - session is trapped in a single vm.

node maintenance (common)

catastrophic failure of a node (exceptional)

Solutions:

Use a load-balancer that supports affinity

Catastrophic failure must be acceptable risk

Maintenance traditionally performed by 'bleeding' sessions from target node.

Stateful Clustering – Affinity - Implementations

Implicit (Memory-based):

Load-balancer simply associates session/[client-ip/ssl-session/connection] etc. with last node to which it sent a corresponding request – BigIP, Pound, Pen, etc...

no changes needed to webapp or web container

awkward if you have a tier of lbs

session:node map may be of limited size/duration (not necessarily a problem)

difficult to inform lb about a session's whereabouts

may not be perfect – e.g. some ISPs may change your IP uderneath you...

Explicit:

Load-balancer or web container injects extra data into request/response that encodes affinity information.

Routing-based – data is appended to session id upon allocation by web container – mod_jk, WebLogic...

requires integration between web container and lb

works as well with a tier of lbs as with a single instance

easier to inform lb about a session's whereabouts

Stateful Clustering - Bleeding

Reconfigure load-balancer to only allow requests bound to existing sessions to continue through to target node.

Other requests that may create new sessions are balanced elsewhere.

Sessions on target node will reach their natural conclusion:

explicit invalidation through e.g. logout

implicit time-out after specified period of inactivity

No new sessions will be created on target node.

When session count hits '0', node may be taken out of service.

Stateful Clustering - Bleeding

Pros:

simple: only a little further complexity added to load-balancer

Cons:

complex, time-consuming management headache

users may keep their sessions alive for an indeterminate time

if site has long-lived sessions becomes an impractical solution

What is a distributable web application?

A compliant web application that also:

declares a <distributable/> tag in its WEB-INF/web.xml

ensures that attributes stored in its HttpSessions are at least one of:

java.io.Serializable

javax.ejb.EJB (local, remote, home, object)

a javax.naming.Context object for the java:comp/env context

javax.transaction.UserTransaction

may receive notification if/when attributes are 'migrated'

does not expect any other data to be distributed by container

Web application enters into contract with web container whereby all state to be distributed is isolated and maintained in a form that can be moved between JVMs by container.

This increases number of strategies available to container for maintaining Quality of Service and High Availability.

Apache Geronimo & WADI

Geronimo - http://geronimo.apache.org

The Apache J2EE Application Server Project

WADI - http://wadi.codehaus.org

“Web Application Distribution Infrastructure”

Originally the design for Geronimo's Web Clustering solution

Now external project, so that standalone web container users might benefit

Plugs into standalone/embedded Tomcat5 and Jetty5

Support for various load-balancer types – extensible – must work with LCD

Spec compatible HttpSession manager with clustering support

Highly configurable

Pluggable subcomponents

WADI technology will feed back into Geronimo EJB clustering solution

WADI will be closely integrated into Geronimo (e.g. multi-tier migration, etc.)

WADI - Goals

Two further simple goals:

'Ctl-C'-ability – painless node maintenance, without session loss

you walk into your web-farm and 'ctl-c' any node...

'bleeding' no longer necessary, live sessions migrated to other nodes

'Kill-9'-ability – no session loss in case of catastrophic failure

you walk into your web-farm and 'kill-9' any node...

WADI - Ctl-C-ability

Further features:

affinity is an optimisation – NOT to be trusted 100%

request relocation – in non-distributable scenario

proxying

redirection (routing lbs - mod_jk)

session relocation – in distributable scenario

peer-to-peer - emmigration/immigration

peer-to-store-to-peer – eviction/'inviction'

pluggable subcomponents:

load-balancer strategies (Simple, Routing/Jk)

eviction policy (relative, absolute, always, never) – more sessions/less resources

shared long-term storage strategy (File – ideal for SAN) - SPoF

streaming strategy (simple, gzip, zip - encryption)

session location (currently multicast, later ActiveCluster)

Request Relocation: Proxying

Useful with 'Memory-based' lbs – BigIP, Pound, Pen, etc..

Request arrives at wrong node.

This node proxies request to correct destination.

Client

Server2

Server1

LoadBalancer

xxx

xxx

Request Relocation: Redirection

Useful with 'Routing-based' lbs – mod_jk, WebLogic

Request arrives at wrong node

This node redirects client back through load-balancer to correct destination (requires integration with lb)

Client

Server2

Server1

LoadBalancer

xxx

xxx

Session Relocation: Migration

Request arrives at wrong node.

Node arranges migration of corresponding session underneath incoming request.

Client

Server2

Server1

LoadBalancer

Tomcat

Jetty

xxx

xxx

Features: Eviction/'Inviction'

Session is evicted to long-term storage after specified period of inactivity.

Subsequent request for evicted session causes it to be 'invicted' onto relevant node under incoming request.

Client

Server2

Server1

LoadBalancer

Store

Tomcat

Jetty

xxx

xxx

WADI – Concurrency/Locking

Spec mandates that request threads may access same session concurrently.

Container may need to serialise session for:

Eviction

Migration

How can container know when it is safe to do so ?

RWLock in every session

Application/Request thread takes non-exclusive R-lock

Container thread takes exclusive W-lock

WADI – 'Ctl-C'-ability revisited

Now have the functionality necessary for 'Ctl-C'-ability.

You ctl-c a node

It begins to shut down gracefully

All sessions are evicted to long-term storage (consider p2p)

Migration requests continue to be serviced during shutdown

Node is out of service

Any break-down in affinity is resolved in a load-balancer specific fashion

NO session loss

When node comes up again, further affinity issues are resolved in same way

'Ctl-C'-ability achieved:

significant complexity – but these features may now be used elsewhere in WADI (eviction)

very low runtime overhead – c.f. in-vm replication (to be discussed)

Very low management overhead

Very high availability of sessions

WADI – 'Kill-9'-ability

Necessary for sites with extreme HA requirements.

Sessions must survive catastrophic failure of their node.

Sessions must be regularly backed up somewhere off-node.

Extremely expensive in terms of cycles and IO.

Linear improvements in QoS become much more difficult.

To be avoided unless extreme HA is a serious requirement.

Traditionally achieved via:

Shared store (DB, FileSystem, Specialist Server, etc) – SPoF - slow/simple

In-VM Replication - fast/complex

Distribution: Shared Store

Session is held in a shared store, loaded into container below incoming request and then saved back to store.

Client

Server2

Server1

LoadBalancer

Store

xxx

xxx

Distribution: In-VM Replication

Primary session held on one node. N replicants are held in-vm on other nodes and kept up-to-date with changes occurring on primary. If primary node fails a replicant is promoted to take its place.

Client

Server2

Server1

LoadBalancer

Server3

xxx

xxx

xxx

xxx

Replication: 1st Generation

Everyone replicates to everyone else

scalability issues

multicast expensive

only realistically need 1-2 'backups'

Replication: 2nd Generation - Partitioned

Manually partition 1st gen. solution

increased management overhead - simply a hack to resolve all-to-all's scalability issues

awkward to integrate well with load-balancer tier

Replication: 3rd Generation

Auto-partitioning

partitioning recalculated as nodes join and leave cluster

awkward to integrate well with load-balancer tier

if session moves, so must replicants

Replication: 4th Generation - WADI

Dynamic, per-Session 'Buddy-Groups'

each session chooses buddies according to Weightings (pluggable composite strategy)

if a session migrates to another node, its buddies do not necessarily have to move too

rock/gravel metaphor...

In-VM Replication – When and What to Send

When?

immediately after any change (lastAccessedTime)

end of request (must serialise requests on session)

end of request group

configurable interval

configurable deviation

What?

delta (invocation/diff)

whole session

Issues

IO/cycles

Object Identity - discuss

In-VM Replication – What will WADI do ?

Buddying:

pluggable strategies: (consider load-balancer, et al.)

per-session granularity (because sessions may be individually migrated)

parameterisable number of buddies chosen via stack of Weightings

same vm/host/subnet/power-supply/room/building?

available resources on target node

ripple amplitude configurable

When?

pluggable strategies: (consider – backup freshness, IO, cycles)

request group

immediate

What?

pluggable strategies: (consider – Object identity, IO, cycles)

delta

whole session

Summary:

A solution should be:

Semantically clean – if your underlying semantics are not absolutely clear, you are building on a shaky foundation. The first issue that WADI resolved was around concurrency in the HttpSession.

Pay-as-you-go – if your requirements are not 'Extreme HA' you should not have to pay the same price simply to get simple cluster management.

Pluggable – container developer cannot predict complete set of deployment environments, so should enable deployer to taylor solution accordingly.

Flexible – implementation should integrate with as many external components as is practicable, and assume LCD functionality from them unless told otherwise, in which case it may optimise.

WADI in Action !

The Demonstration

<distributable/> Gotcha's

Only HttpSession is distributed

Object Identity issues should be reviewed

Session/Attribute listener notifications may appear assymetric

Session size may become more of an issue – normalise your sessions

Expensive activation listeners may become an issue – discuss

Consider other mechanisms for distributing common data rather than putting copies into every session.

If you are using HttpSession attribute as a cache, consider making cached data transient and storing the smaller query that populated the cache – exceptional failover will be slow, but common replication, fast.

Understand what the container is doing and work with it !

Questions & Answers

jules@coredevelopers.net

http://wadi.codehaus.org

Why use an HttpSession?

If you have per-user presentational state and want to:

not force it into your business tier

not manage explicit tracking of session/state across http requests yourself

manage it in a standard way

make use of other standard features:

session/attribute/binding/activation listeners

leverage container's support for this type (specifically distribution)

Other Stuff

show Jk integration code

compare & contrast other distributable webapp containers ?

Jetty DSM

Thomas'

Filip

Ones shipping with Tomcat that require <distributable/>

ditto – that don't

session may only be in one place at once – ramifications

fix slides outline

2003-07-14T11:32:17