AWS, Azure & GCP · Cloud across providers
How does auto-scaling work and what should drive it?
Short answer
Auto-scaling adds or removes capacity based on metrics, schedules or queue depth within minimum and maximum bounds. Good policies use a leading signal, add cool-downs and scale in gradually.
Auto-scaling matches capacity to demand. It needs a metric, a target or threshold, and bounds. The hard part is choosing a signal that reacts before users feel pain and avoiding oscillation when the system scales up and down repeatedly.
Signals and policies
- Target tracking: keep a metric near a target, such as 60 percent CPU or a request count per instance
- Step scaling: add capacity in steps when thresholds are crossed
- Queue depth: better than CPU for workers because backlog reflects real demand
- Scheduled scaling: predictable traffic peaks such as business hours or campaigns
- Predictive scaling: learns patterns, but still needs safe bounds
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60Scale-up should be fast and scale-in slow. Add a cool-down so new instances can warm up, and remember that databases, caches and downstream APIs may not scale at the same rate. Autoscaling without backpressure can turn a traffic spike into a dependency outage.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1Why is queue depth often better than CPU for worker autoscaling?
- 2How do cool-downs prevent scaling oscillation?
- 3What limits autoscaling in a stateful service?
- 4How would you prepare for a known traffic spike?
Say it in the interview
Auto-scaling adjusts capacity within minimum and maximum bounds based on a signal. I prefer a leading signal such as queue depth or request concurrency over raw CPU, and I scale out quickly and in slowly. I also check that downstream dependencies can absorb the extra load, because autoscaling without backpressure just moves the failure.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live AWS, Azure & GCP mock interview and get scored feedback on depth, clarity and confidence.