Module · DevOps Engineer
Incident Response
Triage, mitigate and learn from production failures - one command at a time, no guessing.
TOPIC 01
Triage Under Pressure
Read the evidence in the right order and write down what you actually know.
What you'll practice
- Triage a CrashLoopBackOff — scenario · intermediate
- Explain the Upstream Timeouts — scenario · advanced
A command you'll meet
kubectl logs apiThe last lines of a crashing container are the incident. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Triage a CrashLoopBackOff
The api pod has been restarting since the last deploy. Find why it dies and write the one-line root cause your incident channel needs right now.
Explain the Upstream Timeouts
Yesterday's application log is full of failed requests. Work out which dependency broke first and write the timeline entry that says so - cause, not fallout.
TOPIC 02
Mitigation & Recovery
Restore service first, then keep the fix - a manifest, a service and a record of both.
What you'll practice
- Roll Forward with a Fixed Manifest — challenge · intermediate
- Bring Traffic Back to the Service — challenge · advanced
A command you'll meet
kubectl apply -f ~/manifests/api.yamlRecover with a file, not with a sequence you remember. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Roll Forward with a Fixed Manifest
The api deployment never got its configuration. Write the missing Deployment manifest with three replicas, apply it and let the cluster agree.
Bring Traffic Back to the Service
The pods are healthy again, but clients still get nothing - the Service in front of them is gone. Recreate it and record every command you ran.
TOPIC 03
Postmortems & Runbooks
The fix is not the end of the incident - the write-up and the runbook are.
What you'll practice
- Write the Postmortem — project · advanced
- Publish a Rollback Runbook — practice · intermediate
A command you'll meet
kubectl get podsNever roll back and walk away. Open the task below for the full walkthrough.
Hands-On Tasks in This Topic (2)
Write the Postmortem
The service is back. The part everyone skips is the write-up. Produce a postmortem with impact, root cause and action items - the three sections every blameless review requires.
Publish a Rollback Runbook
The next incident starts at 3am and the person on call will not be you. Write the rollback runbook while you still remember the commands.