D · GovernanceAdvanced55 min read
10. Rollback, recovery & incident response
CI/CD Pipelines · 48 pages source format
Ship knowing how to un-ship. Revert commits, redeploy previous artifact, feature flags — optimize mean time to recovery (MTTR).
What you'll learn
- Revert vs fix forward
- Deploy previous artifact
- Runbook format
- MTTR
Rollback runbook
Step-by-step: detect bad deploy, revert commit or redeploy tag, verify health, communicate.
Do this now
Write ROLLBACK.md — target <15 min recovery for your lab app.
Clear?
Revert commit deploy
git revert + push triggers pipeline redeploys last good state.
Do this now
Simulate bad deploy (break production config). Revert. Confirm recovery.
Clear?
Artifact promotion (awareness)
Build once, promote same artifact staging → prod. Avoid rebuild drift.
Do this now
Sketch promotion flow in PIPELINE.md diagram.
Clear?
Post-incident template
What broke, timeline, root cause, action items — blameless.
Do this now
Add post-incident template to ROLLBACK.md from simulated drill.
Clear?