Kubernetes · Orchestration under pressure
What should you monitor in a Kubernetes cluster?
Short answer
Monitor application symptoms, then the layers beneath: pod restarts, resource usage, node pressure, control-plane health, saturation and storage. Alert on user impact, not every noisy metric.
metrics-server provides basic CPU and memory for HPA, but it is not a full monitoring system. Production usually runs Prometheus or a managed equivalent, node-exporter for host metrics, kube-state-metrics for object state, and instrumentation for application latency, errors and throughput.
- Track the golden signals: latency, traffic, errors and saturation
- Watch restart counts, pending pods, OOMKills and evictions
- Monitor node conditions, disk pressure and allocatable capacity
- Alert on SLO burn rate rather than raw CPU spikes
- Use kube-state-metrics for desired versus available replicas
groups:
- name: api.rules
rules:
- alert: ApiHighErrorRate
expr: sum(rate(http_requests_total{job="api",status=~"5.."}[5m])) / sum(rate(http_requests_total{job="api"}[5m])) > 0.05
for: 10m
labels:
severity: page
annotations:
summary: "API error rate above 5 percent"Design dashboards by audience. Service owners need request rate, latency, errors and saturation. Platform teams need node capacity, control-plane health, API latency and admission webhook failures. Keep alert routing aligned with ownership so pages reach someone who can act.
Follow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1What is the difference between metrics-server and Prometheus?
- 2Which metrics would you use for a Kubernetes SLO?
- 3How do you detect a node that is about to run out of disk?
- 4What would you page on versus what would you put on a dashboard?
Say it in the interview
I start with user-facing signals: latency, errors, traffic and saturation. Under that I watch pod restarts, OOMKills, pending pods, node pressure, control-plane health and storage. metrics-server is only for basic autoscaling metrics, so I use Prometheus, node-exporter and kube-state-metrics for real observability. I page on sustained SLO impact rather than every individual restart or CPU spike.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live Kubernetes mock interview and get scored feedback on depth, clarity and confidence.