System Design · Architecture under constraints
How do you scale an application horizontally?
Short answer
Run multiple interchangeable instances behind a load balancer, remove per-instance state, externalize sessions and files, and make the data layer scale separately. The database usually becomes the next bottleneck.
Horizontal scaling adds more instances rather than a larger machine. It gives redundancy, allows rolling deploys and can follow demand elastically. It only works if a request can be served by any instance, so the first job is removing local state.
What must move out of the instance
- Session data, moved to a shared store or signed into a token
- Uploaded files, moved to object storage
- In-memory caches that assume a single process, replaced with a shared cache or a per-instance cache that tolerates misses
- Background jobs, moved to a queue with workers that can run anywhere
- Local scheduled tasks, replaced by a single leader or a distributed scheduler
Once the tier is stateless, add instances behind a load balancer and use readiness checks so new instances receive traffic only when warm. Set connection pool sizes with the total instance count in mind; ten instances each opening a hundred connections can overwhelm a database that handled one instance with a hundred.
The data tier scales differently. Read replicas absorb read traffic, caches reduce repeated reads, and sharding or partitioning increases write capacity. Each step adds consistency and operational cost, so introduce it only when measurements show the database is the bottleneck.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1What breaks first when you add ten application instances?
- 2How do you handle background jobs in a horizontally scaled service?
- 3Why can connection pooling become a scaling problem?
- 4What metrics would you use for autoscaling?
Say it in the interview
I make the application tier stateless first by externalizing sessions, files and background work, then put interchangeable instances behind a load balancer. I resize connection pools for the total instance count and add readiness checks so traffic only reaches warm processes. After that, the database is usually the bottleneck, so I reach for replicas, caching and partitioning in that order based on measurements.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live System Design mock interview and get scored feedback on depth, clarity and confidence.