When a sev-1 hits, nobody reads a runbook cover to cover. Teams that recover fast follow the same rough sequence, and it is worth knowing before you need it.
1. Detect before users report it
The clock starts when monitoring notices, not when the first ticket arrives. If your users are the alerting system, you already lost the first ten minutes. External probes on real user paths, checking every 15 to 60 seconds, are what shrink this gap.
2. Triage: is it real, and how bad?
One person confirms the signal against a second source. A monitor can be wrong, a region can be partitioned, a deploy can be mid-rollout. Two minutes spent confirming beats thirty minutes chasing a false positive, but do not let confirmation turn into a meeting. The triage question is simple: are users affected right now?
3. Mitigate first, diagnose second
This is the part most teams get backwards. The instinct is to find the root cause while the site burns. Experienced on-calls do the opposite: rollback the deploy, fail over, restart the hung service, shed load. Whatever restores service fastest, even if it hides the cause. You can dig through logs after the graph goes green.
4. Communicate early, in one place
While mitigation runs, someone posts to the status page. Not a postmortem, just: we know, we are on it, updates here. Every minute users spend guessing is a minute they spend writing support tickets and losing trust. A public status page cuts inbound support load during the exact minutes your team can least afford it.
5. Resolve and verify
Service restored is not the same as incident over. Watch the monitors for another few minutes. A fix that flaps is worse than no fix, because it burns the credibility of your next "resolved" update.
6. Postmortem without blame
Within a day or two: timeline, contributing factors, what made it worse, what made it better, and one to three follow-up actions with owners. The question is never who broke it, it is why the system let it break. Most high-severity outages trace back to boring causes: a deploy, a config change, a cert, a disk that filled up over three weeks while nobody watched.
The pattern underneath
Fast teams are not smarter. They have tighter detection, a rollback that actually works, a status page ready before it is needed, and postmortems that fix systems instead of blaming people. Every one of those is buildable before the incident, and none of them during it.