Load balancing and high availability

One of the benefits of working with people smarter than you is the chance to never stop learning. My office mate, Bernie Ács, is an extraordinary source of experience and wisdom on production systems. Just right after coming back from vacation, he started unfolding a virtualized solution for high availability and load balancing of services. It is not me the one who should disclose the solution, but I just want to mention a piece of the puzzle that I found very interesting: ldirectord. In the words of the ldirectord developers:

ldirectord is a daemon to monitor and administer real servers in a LVS cluster of load balanced virtual servers. […]  A LVS cluster consists or one or more virtual services each of which may have zero or more real servers. The IP address of a virtual service is what end-users connect to and is typically advertised over DNS. When a connection is made to a virtual service, it is allocated a real server, and all packets for this connection are forwarded to this real server. A more detailed overview of LVS can be found here.

What does this mean? Let me sketch a simple example. Imaging you have a web server that hosts your new cool site providing content to the web. To get to your site people navigate to a particular server name that, via DNS, gets resolved into the particular IP address where a service runs providing your cool content over HTTP. So far so good. But what happens when a lot of people starts coming to visit your cool and now more and more popular site? Maybe somebody mentioned your site at Slashdot…

The answer: Your server starts getting slow to a crawl, or it even goes down because it cannot serve content to all the incoming requests, and people do not want to wait for your cool content any longer and they eventually end moving on and, eventually, forget your site.

To avoid getting to that extreme, ldirectord comes into the picture. The idea is simple, make people hit a high available virtual server whose only purpose in life is to forward incoming requests to a farm of real servers, each of which providing your cool content—yes, you can see them as clones of your original server.