Load Balancing Algorithms

One-Liner

The different strategies a load balancer can use to distribute traffic to backend servers.

What It Is

The set of rules that a load balancer uses to select which server should receive the next incoming request.

Why It Exists

To provide different ways of distributing traffic based on the needs of the application, such as ensuring even distribution or maintaining session persistence.

How It Works

Round Robin: Distributes requests to servers in a circular order. Simple but doesn’t account for server load.
Least Connections: Sends the next request to the server that currently has the fewest active connections. More intelligent than Round Robin.
IP Hash: The client’s IP address is used to determine which server receives the request. This provides a form of “sticky session,” ensuring that a user is consistently sent to the same server.
Weighted: Servers are assigned different weights based on their capacity, and the load balancer distributes traffic accordingly.

Tradeoffs

Round Robin vs. Least Connections: Simplicity vs. better load distribution.
IP Hash vs. other methods: Provides session persistence but can lead to uneven load distribution if some IP addresses send many more requests than others.

Failure Modes

Uneven load distribution with IP Hash: A few power users from a single IP can overload one server while others are idle.
“Thundering herd” with Least Connections: If a new server comes online, it will initially have zero connections and may be immediately overwhelmed by a flood of new requests.

Interview Traps

Only knowing about Round Robin.
Not being able to explain the pros and cons of sticky sessions.

Real-World Usage

Different algorithms are used depending on the application’s requirements for session state and server capacity.

Anti-Patterns

Using IP Hash for an application that does not require session persistence, as it can lead to suboptimal load distribution.

Layer 4 vs. Layer 7 Load Balancing
Session Persistence
Health Checks