Load Balancing

Once you have multiple servers, something needs to decide which server handles each request. That's a load balancer.

Why You Need One

A load balancer distributes incoming traffic across multiple backend servers, preventing any single server from becoming a bottleneck, and it removes unhealthy servers from rotation automatically via health checks.

Common Algorithms

Round robin cycles through servers evenly. Least connections routes to the server currently handling the fewest active requests. Weighted routing sends more traffic to more powerful servers. IP hash routes the same client consistently to the same server, useful for session affinity.

Picking the wrong algorithm for your traffic shape is a subtle failure mode: round robin assumes requests are roughly equal cost, which breaks down if some requests are far more expensive than others (say, a report generation endpoint next to a simple lookup). In that case, least connections adapts better because it accounts for how busy each server actually is, not just how many requests it's received.

Layer 4 vs Layer 7

Layer 4 load balancers route based on IP and port, and are fast but blind to request content. Layer 7 load balancers inspect the actual HTTP request — path, headers, cookies — allowing smarter routing, at the cost of more processing overhead.

Health Checks and Failover

A load balancer is only as good as its health checks. Servers that fail to respond correctly to periodic pings are automatically pulled out of rotation, and traffic is redistributed to healthy instances — this is a major part of how systems stay available during partial failures.

A Single Point of Failure, Solved

A lone load balancer is itself a single point of failure. Production systems typically run redundant load balancers behind a floating IP or DNS-based failover, so the load balancing layer doesn't become the very bottleneck it was meant to eliminate.

Client-Side vs Server-Side Load Balancing

Most interview answers assume a server-side load balancer sitting in front of your services, but it's worth knowing client-side load balancing exists too — the calling service maintains a list of healthy instances and picks one itself, common in service-mesh architectures. Mentioning it briefly shows awareness beyond the textbook default, though a server-side load balancer is the safer answer to lead with.