System Design · Architecture under constraints
How does load balancing work and how do you choose an algorithm?
Short answer
A load balancer distributes traffic across healthy backends. L4 routes by connection, L7 routes by HTTP content. Algorithms range from round robin to least connections and consistent hashing.
A load balancer sits between clients and a pool of servers. It performs health checks, removes unhealthy instances, and selects a backend for each connection or request. It also terminates TLS, enforces connection limits and can support canary or blue-green routing.
Layer 4 versus layer 7
- L4 sees TCP or UDP and forwards connections without parsing HTTP, which is fast and protocol-agnostic
- L7 parses HTTP and can route by path, header, cookie or method
- L7 enables retries, request-level metrics and content-aware canaries at a higher CPU cost
- L4 cannot make decisions based on a URL or an authorization header
Common algorithms
- Round robin: simple and fine when requests have similar cost
- Least connections or least outstanding requests: better when request cost varies
- Power of two choices: sample two backends and pick the less loaded one, with low coordination
- Consistent hashing: keeps a key on the same backend when the pool changes, useful for caches
- Weighted routing: accounts for heterogeneous hardware or canary traffic
Health checks need to reflect real serving ability. A TCP connect check can pass while the process is out of database connections. Prefer an application endpoint that verifies critical dependencies, with separate liveness and readiness semantics, and make sure the load balancer itself is redundant.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1When would you use least connections instead of round robin?
- 2What is the difference between liveness and readiness?
- 3How do you avoid the load balancer becoming a single point of failure?
- 4Why can sticky sessions be harmful?
Say it in the interview
A load balancer spreads traffic over healthy backends and removes failed ones. I choose L4 when I only need fast connection distribution and L7 when routing depends on HTTP details. The algorithm follows the workload: round robin for uniform requests, least outstanding requests when cost varies, and consistent hashing when cache locality matters. I also make the balancer redundant and health-check real serving ability.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live System Design mock interview and get scored feedback on depth, clarity and confidence.