This document is a compendium of my notes on the different load balancing strategies that one can employ for their application.
Intermediate Load Balancer
- This refers to the strategy of load balancing by using an independent external load balancing device. The device could be a hardware load balancer or a box running a software load balancer.
- All incoming traffic is directed to this device
- This device based on rules then distributes the traffic to the backend infrastructure
- The device also maintains a hearetbeat with each Server and if it finds any server is not responding, based on specific configuration, it redistributes the load amongst the balance devices
- The device can perform load balancing at Layer 7, Layer 4, Layer 3 (Route-path), Layer 2 (Bridge-path)
- All connections/packets pass through the load balancer on their way in and out
- Advantages
- Very easy to setup
- Easy failover. If a machine goes down all logic for failover resides in the load balancer which automatically handles the redistribution
- Adding nodes is quite easy too
- Fairly accurate load distribution since all traffic passes through the load balancer
- Ability to configure sticky sessions, where in a client is sent to a single destination for the duration of their session (For extreme accuracy - may require Layer 7 inspection - which can be expensive)
- Disadvantages
- The load balancer now becomes a single point of failure. Hardware load balancers are quite resilient so this may or may not be an issue depending on your redundancy requirements
- Hardware load balancers can be expensive
- Load Balancers needs to support a throughput equal to the sum total traffic of the cluster
- If you use Layer 7 based sticky sessions the cost and latency increase, since every packet has to be inspected all the way till Layer 7 in the Load Balancer and then reconstructed and sent back on the wire
- At the minimum each packet will be inspected until Layer 3 or Layer 4 (since most common load balancers base their rules on these layers) which results in a certain amount of processing requirements
- A packet not only has to be inspected, but then subsequently its destination address also has to be re-written before sending it across the wire (additional CPU requirement)
Wackamole with Round Robin DNS (RRD)
- Wackamole (or mod_backhand) is an open source utility which provides the failover functionality in a RRD based environment without the need for any intermediate load balancer
- All incoming traffic to the cluster is load balanced through round robin DNS
- All nodes run wackamole which continually checks the heartbeat of all the servers in the cluster
- If any node is non-responsive, wackamole kicks-in and automatically redistributes the IP addresses of that machine to the other live machines in the cluster
- Advantages
- No intermediate load balancer required
- No packet inspection, added latency or single point of failure
- Low cost
- Disadvantages
- DNS RRD does generally result in equal load balancing, however in some cases, where a set of clients are sending far more requests than all other clients, the load balancing may not be equal
- Configuring differential load balancing between the nodes requires proportionately distributing IP addresses across those nodes and within the DNS entries
- Load distribution may not be equal during a downtime, since the ip addresses of the machine that is down are divided amongst the remaining hosts, and the number of ip addresses may not be a integral multiple of number of hosts. For eg, if we have 4 machines, each with two ips, and if one machine goes down, then 2 of the remaining 3 machines will take over the two ip addresses, resulting in unequal load distribution. This may not be a huge issue since downtimes should ideally be cured quickly. Additionally one can minimize this effect by planning their ip distribution and number of ip addresses
- Cannot configure granular sticky sessions behavior. Typically sticky sessions is automatically achieved whether you like it or not, since the end client may likely cache the ip address it receives through RRD and continue to send all its requests to that IP
Network Load Balancing (NLB) with Multicast/Broadcast
- NLB is a windows based load balancing module which provides load balancing functionality without the requirement of an external load balancer
- In NLB, all machines in the cluster receive all packets / connections. This can be engineered in several different ways - multicast, broadcast, switch port cloning etc. This is done at a layer 2 level within the network equipment
- The packets/connections simply reach the network adaptor of each machine in the cluster
- The NLB module sits right on top of the adaptor driver and receives the packet
- Based on the source ip address of the client and/OR the port number of the client, an algorithm in the NLB module of each server in the cluster decides whether they should pickup this connection or not
- The algorithm guarantees that only one machine in the cluster will pick up the packet. Think of this as an agree-upon horizontal partitioning strategy. For instance a trivial implementation could be, that in a two machine cluster - machine 1 picks up all connections with odd ip addresses and machine 2 picks up all connections with even IP addresses. This strategy is fool proof and does not require any communication between the machines at all. The actual algorithm is a randomization algorithm that uses more than just the client ip address (based on configuration) to determine whether it should pick up that packet
- If the NLB module figures that this packet should be picked up by it, it simply passes it through the stack upwards. If not it simply discards the packet at this stage itself
- NLB supports differential load balancing configurations wherein one can specify unequal distribution of load between machines in a cluster and its algorithm will automatically take care of it
- NLB also maintains a heartbeat between machines and if any machine is found to be down, it will automatically redistribute the load amongst the remaining machines
- NLB also supports client_affinity (another name for sticky sessions) where in connections originating from a single IP will be sent to a single source. This can screw up if one is using a reverse-proxy in front of the cluster, since all connections will appear to originate from a single ip address. One should not enable client_affinity if one has a reverse proxy. In most cases if you have a reverse-proxy, the reverse-proxy itself can perform relevant load balancing, so NLB would be redundant
- Advantages
- No hardware load balancer required
- No packet inspection, added latency or single point of failure
- Disadvantages
- Since all packets are sent to all machines, network traffic in terms of the pipeline to the server will be utilized more than required. This usually does not pose any problems, since most servers have Gig-E ethernet adaptors and typical machines cannot saturate Gig-E pipes anyways
- All machines will waste some CPU cycles on inspecting packets and discarding them. On a per packet basis, the amount of CPU consumed by an NLB module (inspect and discard) is lesser than the cpu consumed by a typical intermediate Load Balancer (inspect, modify and forward). However it must be noted that this inspect-and-discard algorithm needs to run on all machines for all traffic. As traffic increases, proportionate CPU overhead required by the NLB module increases on each node in the cluster. This can seem sub-optimal, however in most circumstances, for traffic ranging upto a few 100 MBps the CPU overhead of the NLB module is not too high (ranging between 6-10% of a single 450 Mhz CPU). Nevertheless this is an extra overhead.
Conclusions
- Of all the load balancing methods, Wackamole with RRD is the most efficient. It has no packet inspection overhead and offers identical redundancy and high availability
Failover strategies
- At failover time, load balancers need to be configured to re-route traffic to the remaining servers in a cluster. This decision cannot be taken lightly especially when there is state involved.
- It is important to decide what set of activities will result in a failover, and what exactly constitutes unavailability
- For instance lets say we have a cluster of web servers
- Some form of a heartbeat protocol exists to determine whether the machine is up or not
- It is important that the heartbeat protocol tests the appropriate service. For instance, it maybe possible that on one of the nodes, the machine is up and working, but the web service is not responding. If the heartbeat service is monitoring the machine, it may not detect downtime of the App server on the machine
- Additionally, what exactly constitutes a downtime. For instance, typical algorithms maybe of the following type -
- Make a request every 5 seconds. If the request fails then make 3 more requests ever 1 seconds. If they all fail then assume the node is down. This ensures that a node is assumed to be down after adequate inspection instead of one single random request which can fail at times. However in this model as you will guess, a node could be down for a total of 8 seconds before the failover process kicks in
- In stateful services it maybe important that the cluster servers share state in some manner. For instance web servers can achieve this by using a central session store, or some clustered session protocol which replicates sessions on each node of the cluster
- In stateful services that involve data stores (eg databases) one needs to be extra careful. Web servers are easy to load balance. One can add and remove servers to and from the cluster without worrying too much. However one cannot do the same with database servers indiscriminately, since database servers use complex replication protocols to ensure that the other servers in the cluster contain the same data. Infact true master-master replication is not really common-place in most RDBMS, and therefore typically most replication architectures are setup as master-slave, wherein if the master goes down, one of the other slaves may have to be promoted to a master. This process of promotion is not trivial since it needs to ensure that the selected slave is the best candidate. Additionally if the original node that went down, comes back online, it needs to now reconfigure itself as a slave, catchup with the lost transactions and then continue to serve queries. This process can require considerable configuration and care, and hence one cannot add nodes or remove nodes from the cluster indiscriminately.
References
|
|
|
