Blameless Postmortems: Turning Outages into Organisational Learning

The previous post was about finding the root cause — but then what? The postmortem turns one expensive outage into learning for the whole organisation: what happened, how big the impact, the timeline, the true cause, how it was fixed, how to prevent a repeat. And its soul is a word that looks simple and is extremely hard to live up to: blameless (the event, not the person).

The soul is blameless: the event, not the person

The same outage, handled by “blame” or by “blameless”, takes a team down two completely opposite cycles:

Why is blame so deadly? Because it scares off the truth. When mistakes are punished, people instinctively hide them, tidy up the timeline, and don’t dare say “actually I saw X at the time but didn’t think much of it” — and to learn the lesson, you need exactly that complete, honest truth. Blameless isn’t “being a pushover, nobody accountable”; it’s safety deliberately designed to obtain the truth.

A person is almost never the root cause

Blameless has an even harder underlying logic: people will make mistakes; that’s a constant; so “preventing people from making mistakes” is futile, and what you should do is “make sure a mistake doesn’t become a disaster”. The classic example — someone deletes the production database with one command:

So a postmortem has an important premise: assume that everyone, in the moment, made a reasonable decision based on the information they had (assume good intentions). Nobody gets up in the morning thinking “let’s bring down the system today”. With that assumption, attention naturally moves from “how could this person be so stupid” to “what system, process, or information gap led a reasonable person to take an action that broke things” — and only the latter can be fixed.

Blameless doesn’t mean unaccountable

To clear up a common misreading: blameless isn’t “nobody is responsible, everyone muddles along”. There must still be clear action items, each owned by someone, tracked to completion — the focus is just on fixing the system, not punishing the individual. And action items must be concrete and executable: “add a confirmation step to the delete command”, “schedule a backup-restore drill monthly” are action items; “everyone be more careful from now on” is not — that just hands the same pain, untouched, to next time.

Reflections

The cost of blame is scaring off learning

The greatest damage of blame doesn’t land on the person being scolded; it’s that it makes “telling the truth” dangerous. Once admitting a mistake carries a price, the whole team starts hiding, tidying up, defending — and what you most need is precisely the unvarnished full truth. So I increasingly see blameless as a very pragmatic design, not a moral posture: you give up prosecuting individuals in exchange for honesty; and honesty is the only precondition for a team to learn anything from failure. It’s the same thing as the psychological safety I’ve always believed in when leading people and along the Tech Leader line — people are honest only when they feel safe; and a dishonest team never learns, however many outages it has.

If one slip can bring down the system, that’s a system problem

I want to amplify this line, because it’s hard enough to be a creed. People making mistakes is a constant, not a variable; since you can’t remove it, effort spent on “preventing people from erring” is wasted. What you should do is make the system resilient to human error — foolproofing, confirmations, least privilege, fast restore. The root cause of the deleted database was never “Alice’s hand slipped”; it was “the system allowed one slip to destroy everything”. That’s exactly the same sentence as the first post‘s “fault tolerance isn’t fault absence”, except this time the fault being tolerated is a human one: a good system assumes people will err, and makes sure erring doesn’t become a disaster.

Not writing the postmortem means the pain was for nothing

The tuition for an outage is very expensive — late nights, apologies, lost trust, error budget burned. Something that expensive, if not converted into a postmortem the organisation remembers and that prevents a repeat, is pure loss. I now treat writing a postmortem as “turning pain into an asset”: it hurt anyway, so at least get a stabler system and a group of people who genuinely learned. And whether that trade goes through depends entirely on the action items being concrete, owned, and tracked — which echoes the previous post‘s “record what you did”: the trail debugging leaves behind is the best raw material for a postmortem. It hurt already; don’t let it pass for nothing.