CI/CD · Ship safely, repeatedly
What is your process for responding to a deployment incident?
Short answer
Detect and declare the incident, assign an incident commander, mitigate first with rollback or a feature flag, communicate status, then investigate and run a blameless postmortem with tracked follow-ups.
The priority during an incident is restoring service, not finding the root cause. Diagnosis can continue in parallel, but the mitigation path should be independent and pre-rehearsed.
- Declare severity and assign clear roles: incident commander, communications lead and operations lead
- Stop the rollout and mitigate with rollback, traffic shift or a feature flag
- Communicate impact and status on a predictable cadence to stakeholders
- Preserve evidence such as logs, deploy markers and metric snapshots
- Confirm recovery with the same signals that detected the problem
- Run a blameless postmortem and track corrective actions to completion
kubectl rollout undo deployment/checkout
kubectl rollout status deployment/checkout --timeout=5m
kubectl get pods -l app=checkoutFollow-ups you should expect
Interviewers use these to check whether the first answer was understanding or memorisation.
- 1What does an incident commander do?
- 2How do you decide between rollback and forward-fix?
- 3What belongs in a blameless postmortem?
- 4How do you keep stakeholders informed without distracting responders?
Say it in the interview
I mitigate first: stop the rollout and roll back, shift traffic or disable a flag. In parallel I declare severity, assign an incident commander and a communications lead, and keep a timeline of what we observed. Once service is restored I confirm recovery with the same metrics, preserve evidence and run a blameless postmortem. Every follow-up has an owner and changes a system or a check, not just a promise to be more careful.
Use this as a model, not a script. Saying it in your own words is what makes it survive follow-up pressure.
Related questions
Reading is step one
Now say it out loud.
Understanding an answer and delivering it under pressure are different skills. Rehearse this topic in a live CI/CD mock interview and get scored feedback on depth, clarity and confidence.