Copy of Load Balancers (on 2022-01-11)

What is load balancing?

Dividing the load of requests among the set of serving resources is called load balancing.

Within the design of a system, load balancing is performed at various layers like:

• Globally arriving requests forwarded to individual data centers.
• Dividing requests coming to a data center among the available web servers, application servers, cache servers, and database servers.

Load balancing helps divide the load evenly among all the resources, efficiently utilize existing resources, reduce user response time, etc.

Load balancing in DNS

We understand that DNS responds with multiple IP addresses for each DNS query. In the lesson on DNS, we discussed that it is possible to do load balancing through DNS while looking at the output of nslookup. DNS uses a trick of changing the order of the IP addresses in response to each DNS query. Thus, different users getting different IPs will result in users visiting a different server to entertain their requests. In this way, DNS distributes the load of requests on different datacenters. Hence, doing global-level load balancing.

DNS can use a simple strategy of round-robin to perform load balancing as shown below:

As shown above, Round-robin in DNS forwards clients to data centers in a strict circular order. Round-robin has the following limitations because of end-user caching:

1. An ISP serving many customers will provide the same cached IP, resulting in uneven load distribution on end-servers.
2. Because the round-robin load-balancing algorithm does not consider any end-server crashes, it will keep on distributing the IP address of the crashed servers until the record owner updates the DNS record and the TTL of the older one expires. Thus, availability can take a hit in that situation.

Despite its limitations, round-robin is still a widely used load balancing algorithm many DNS service providers use. Other algorithms like fewer connection load balancing also exist.

The need for load balancers

DNS plays a vital role in balancing the load but suffers from the following limitations:

• Small size of the DNS packet (512 bytes) isn’t enough to include all possible IP addresses of the servers.
• There is limited control over the client’s behavior. Clients may select arbitrarily from the received set of IP addresses. Some of the received IP addresses may belong to busy data centers.
• Clients cannot determine the closest address to establish a connection with. Although, DNS geolocation and anycasting-like solutions can be implemented. They are not trivial solutions.
• In case of failures, recovery can be slow through DNS because of caching mechanism.

To solve these problems, we need another layer of load balancing in the form of Load balancers (LB). LBs behave like a reverse proxyReverse proxy is a component at the edge of the server-side network that sits between the server and the outside world. It can provide features like privacy, security, and caching, etc. and does its best effort to divide incoming requests among the pool of available servers. Incoming clients’ requests connect to the LB that uses a Virtual IP address ( VIPVIP).

A load balancer may not be a requirement if your service entertains a few hundred or even a hundred thousand requests. However, for increasing clients requests load ...