Skip to content
System Design

Load Balancing

Scope

Techniques for distributing incoming client requests across multiple computing resources, such as application servers or databases.

Why This Topic Exists

Load balancing is a foundational strategy for achieving scalability and high availability. By distributing traffic, it prevents any single server from becoming a bottleneck and allows systems to gracefully handle failures by redirecting traffic away from unhealthy servers.

Core Tradeoffs

  • Layer 4 vs. Layer 7: The speed and simplicity of transport-layer balancing versus the intelligence and flexibility of application-layer balancing.
  • Algorithm Choice: The simplicity of Round Robin vs. the more even distribution of Least Connections vs. the session persistence of IP Hash.
  • Session Persistence (Sticky Sessions): Can simplify stateful applications but leads to uneven load distribution and complicates server fleet management.
  • Centralized vs. Decentralized: A dedicated hardware load balancer versus client-side or sidecar proxy load balancing.

Common Failure Modes

  • The Load Balancer as a Single Point of Failure: Without a high-availability (HA) setup, the load balancer itself can bring down the entire system.
  • Improperly Configured Health Checks: The load balancer may fail to detect an unhealthy backend server, continuing to send traffic to it (“black-holing”).
  • Uneven Traffic Distribution: Sticky sessions or poorly chosen hashing keys can lead to some backend servers being overloaded while others are idle.
  • Upstream Cascading Failures: A slow upstream service can cause connection pools on the load balancer to become exhausted, impacting all traffic.

Interview Signals

Strong candidates will not just list algorithms but will discuss the practical implications of choosing between Layer 4 and Layer 7 load balancing. They will also bring up operational concerns like health checks, high availability configurations (active-active vs. active-passive), and the pros and cons of SSL termination at the load balancer.

  • DNS
  • Scalability
  • Reliability
  • Service Discovery