Load Balancing Algorithms Explained: When to Use Which Algorithm Simplified

You have one server. Traffic grows. It starts crashing at odd hours. You add a second server. The problem is not solved. Now you have two servers and nobody deciding which request goes to which one. Half your users get errors because both servers are getting traffic but neither one knows the other exists. The fix is a load balancer, and the interesting part is how it decides where to send each request.

The Problem You Are Actually Solving

Every server has a limit. CPU, memory, disk I/O, network bandwidth. At some point, a single machine cannot handle all your traffic. The fix is to add more machines and split the traffic between them. But who decides which request goes where? That is the load balancer. It sits in front of your servers and routes each request intelligently. It knows which servers are healthy. It knows which ones have capacity. The algorithm it uses determines how fair that distribution is.

Round Robin: The Simplest Approach

Round robin is exactly what it sounds like. The load balancer keeps a list of servers and cycles through them in order. Server A gets request 1. Server B gets request 2. Server C gets request 3. Then the cycle starts over: A, B, C, A, B, C.

There is no state to maintain between requests. The load balancer does not need to remember anything about previous requests. It just increments a counter and sends the next one to the next server. This makes round robin very fast and very simple to implement.

The problem is that round robin assumes all servers are equal. If Server A has twice the memory and CPU of Server B and Server C, it will still get exactly one third of the traffic. The powerful server sits underutilized while the weaker ones strain under the same load. Round robin is blind to server capacity. It also has no awareness of current load. A server processing a long file upload still gets the same number of new requests as one sitting idle.

Round robin works fine when your servers are identical and requests are uniform. Most small deployments start here.

Weighted Round Robin: When Servers Are Not Equal

You have three servers but one is newer and twice as powerful. Weighted round robin lets you express that. You assign each server a weight, say Server A gets weight 2, Server B gets 1, Server C gets 1. The load balancer cycles through them but Server A gets twice as many requests as the other two combined.

Weights are static. You set them when you configure the load balancer and update them when your infrastructure changes. The practical use cases come up more often than you might expect. When you add a new server with more capacity, you give it a higher weight while traffic gradually increases. When you are transitioning between hardware generations, you weight the newer machines higher as you deprecate the old ones.

The limitation is the same as basic round robin: weights do not adapt to current load. A server under a long-running request still gets its proportional share of new requests regardless of how busy it already is.

Least Connections: The Busy Server Problem

Round robin breaks down when request times vary. Imagine Server A handles simple API calls that return in milliseconds. Server B handles file uploads that take 30 seconds. After 30 seconds, Server A has processed thousands of requests while Server B is still working through the same batch. Round robin does not know this and keeps sending equal traffic to both.

Least connections fixes this. The load balancer tracks how many active connections each server has at any given moment. When a new request comes in, it goes to the server with the fewest active connections. A server that is already swamped does not get new work until it catches up.

This is the right algorithm for workloads where request duration varies significantly. WebSocket connections that stay open for hours. Long-polling requests. File processing jobs. Database query endpoints where some queries run in milliseconds and others take minutes. In all these cases, the server that looks available may actually be the busiest one, and least connections handles that correctly.

The cost is state. The load balancer has to track every open connection for every server. This is not expensive in modern hardware but it means the load balancer cannot be stateless. You lose the operational simplicity of round robin.

Least Response Time: Speed and Load Together

Least connections looks at queue length. Least response time looks at how fast the server actually responds. A server with fewer connections might still be slower because it has older hardware or less available memory. The load balancer tracks the average response time for each server and factors that in.

The formula is straightforward: the server with the lowest (active connections divided by average response time) wins the next request. A server with 5 active connections that responds in 10ms on average scores better than a server with 3 active connections that responds in 100ms on average. The algorithm balances how busy the server is against how fast it is.

Major CDN providers use this approach at the global level. When you load a website, you are being routed not just to a server that has capacity but to the one that responded fastest to previous requests from your region. This happens transparently before the request even reaches your origin servers.

IP Hash: Same User, Same Server

Sometimes you need a specific user to keep hitting the same server. If your application stores session data locally on each server rather than in a shared database, then user A must always reach Server B or their session disappears. IP hash uses the client IP address as input to a hash function, which maps it to a specific server. The same IP always lands on the same server.

The problem is that IP hash creates session affinity rather than solving it. If Server B goes down, every user mapped to Server B loses their session. You have fixed one problem by creating a new one. The real solution is to store session data in a shared database or Redis cluster, so any server can handle any user. IP hash is a band-aid over an architectural problem.

It also breaks down behind large NAT gateways. Thousands of users appear to come from the same IP address. They all get routed to the same server, which becomes overloaded while others sit idle. This is common on mobile networks and some enterprise proxies.

Random: Simpler Than Round Robin

Picking a server at random sounds too simple to be useful. Under sufficient load, it is actually quite effective. With enough requests, random distribution converges toward equal distribution. The mathematical term is that random sampling approximates the target distribution given enough samples.

The advantage over round robin is conceptual simplicity. There is no ordering, no state, no counter to maintain. Each request is independent. The load balancer does not need to track anything between requests.

A minor improvement is two random picks. The load balancer selects two servers at random and sends the request to the one with fewer active connections. This reduces the chance of accidentally routing to an overloaded server. It still requires tracking connection counts like least connections but makes the initial selection without state.

Conclusion

Round robin is simple and stateless, good for identical servers with uniform requests. Weighted round robin adds capacity awareness. Least connections handles varying request durations correctly. Least response time adds speed to the equation. IP hash solves session affinity but masks a deeper problem. Random works better than it sounds under load. Health checks are not optional, they are what make every algorithm safe to run.

At scale, you layer these approaches. DNS for geographic rough cuts. Layer 4 for raw throughput. Layer 7 for intelligent routing. Anycast for global distribution and attack absorption. The algorithm matters less than knowing what your workload needs and measuring whether you are hitting your targets.