For a long time I treated reliability the way most teams do, as plumbing. It was something you got to after the real work, it lived in a runbook nobody read until something broke, and it competed for attention against the work that showed up in release notes. Reliability never won that competition, because it is invisible when it is working. Nobody opens the app and thinks, the database held up beautifully today. They only notice when it does not, and then they notice everything at once.
The shift that changed how I operate was simple to state and hard to live by. Reliability is a feature, and it deserves the same treatment as any other feature: an owner, a roadmap, a backlog, and a place in planning where it competes on equal terms. Uptime is not a property of some infrastructure layer off to the side. It is part of the product. Here is what actually changed when I stopped pretending otherwise.
The problem with reliability as a background concern
When reliability is everyone's job, it is no one's job. That is the trap, and it is a structural one, not a question of how much your engineers care. Caring is not the constraint. Attention is. If reliability work has no owner and no slot on the roadmap, it surfaces only as interruptions, and interrupt-driven work is the worst way to build anything durable.
What that looked like for us, in the years before the change, was a familiar pattern. An incident would happen. We would scramble, fix it, write a postmortem, and generate a list of action items. Those action items would go into a backlog where they would sit, outranked by feature work every single planning cycle, until the same class of incident happened again and we rediscovered the action items we had already written. We were paying for the same outage twice, and the second payment was the morale of the people who had warned us.
The deeper issue is that you cannot prioritize what you cannot see. Reliability had no scoreboard. We had uptime numbers somewhere, but they were not in front of the people making roadmap decisions, so reliability lost every argument by default. It was not that we chose features over stability. It was that stability never got to make its case.
Give reliability an owner and a roadmap
The first concrete change is to put a name on it. The move that matters is making reliability something one person owns, the way a product manager owns a feature area, rather than a responsibility spread so thin that nobody actually carries it. That owner maintains a reliability roadmap. It has themes, it has quarters, it has things they are deliberately not doing yet. It goes through the same planning and prioritization as everything else, and it gets real time set aside rather than a vague hope that someone will find a spare afternoon.
The discipline that makes this real is giving reliability a genuine share of capacity, not the scraps left over after features. Not as a tax extracted reluctantly, but as a planned investment with expected returns. Some seasons that share is smaller, right after you have paid down a lot of risk. Some seasons it is larger, when you are scaling into a new order of magnitude. But it is always on the board, always defended, and never the thing that silently gets cut when a deadline tightens.
Giving reliability an owner also fixes the accountability gap. When something is everyone's responsibility, postmortems produce action items addressed to no one. When reliability has an owner, those items have a home, a prioritized queue, and someone who answers for whether they actually got done. That turns the postmortem from theater into a backlog that actually gets worked.
Decide what reliability means before you promise it
You cannot manage uptime as a feeling. So we did the unglamorous work of defining service level objectives for the parts of the product that matter most, and we were honest that not every surface deserves the same target. The core flows in Atlas, the things a customer touches every hour like opening a task or sending from the inbox, carry a higher objective than a background export that can retry. Treating everything as equally critical is the same mistake as treating nothing as critical. You spread effort evenly and protect nothing well.
Defining objectives forced a conversation we had been avoiding, which is how reliable we actually intend to be, and what we are willing to spend to get there. Every nine you add costs more than the last one, and the jump from 99.9 to 99.99 percent is not a tuning exercise, it is an architecture and a culture. Saying the number out loud made the cost legible and the tradeoffs explicit. That is the point of an SLO. It is not a promise to be perfect, it is a budget for imperfection that everyone agrees to in advance.
The error budget that falls out of an objective is the most useful management tool we adopted. When we are comfortably inside budget, we ship fast and take more risk, because we have earned the room. When we burn through it, feature work pauses and reliability work takes priority automatically, without anyone having to win a political argument in the moment. The budget makes the decision, which means the decision gets made the same way under pressure as it does in calm. That consistency is most of the value.
Build the systems that make reliability cheap to maintain
Owning reliability is not mostly about heroics during incidents. It is about the boring, compounding investments that make the system hold itself up so people do not have to. A few of them did more for us than anything else:
- Observability before features. If you cannot see it, you cannot operate it. We instrument new surfaces before we ship them, so that the first time something goes wrong we are reading a dashboard instead of guessing in the dark.
- Graceful degradation by design. When a dependency is slow, the right behavior is to serve a degraded but working experience, not to fail the whole page. A slow CRM lookup should not take down the task view next to it.
- Automated rollback. The fastest fix for most incidents is undoing the last change. Making rollback a one-click, low-anxiety operation turned a class of thirty-minute outages into three-minute ones.
- Load testing against real shapes. Synthetic traffic that does not look like real usage tells you nothing. We test against the actual access patterns we see in production, including the ugly ones.
None of these are exciting. That is the point. The work that keeps a system up is almost always unglamorous, which is exactly why it loses to features when reliability has no owner and no roadmap. Give it both, and the boring work finally gets done, and the boring work is what scaling actually requires.
Run incidents to learn, not to assign blame
The other half of treating reliability as a feature is how you behave when it fails, because it will fail. No system at scale stays up forever, and the teams that recover fastest are not the ones with the fewest incidents, they are the ones that have made incidents safe to handle well. The single most important habit is making the postmortem blameless in practice and not just in the document template. The question is never who broke it. The question is what in the system made it possible to break it that way, and what would have caught it sooner.
This matters because the alternative is silence. In a blame culture, the rational move during an incident is to minimize your exposure, which means people hesitate to act, hesitate to admit what they changed, and hesitate to share what they actually saw. Every one of those hesitations adds minutes to recovery and removes information from the postmortem. When it is genuinely safe to say "I deployed the change that caused this," recovery gets faster and the lessons get real, because the person closest to the failure stops protecting themselves and starts explaining what happened.
A clear incident command structure helps too, so that during an outage there is one person coordinating and a known way to escalate, rather than everyone involved guessing in the same channel at once. The structure is not bureaucracy, it is the thing that keeps a high-pressure moment from turning into chaos. And it is worth measuring recovery, not just frequency, because how fast you come back from a failure says more about your engineering than how often you fail. Recovering in four minutes is a different reliability posture than recovering in forty, even with the same number of incidents on paper.
Reliability is a feature your customers feel
The reason all of this is worth the organizational effort comes back to the customer. They do not see your architecture. They will never know about the rollback you automated or the dependency you isolated. But they feel every outage, immediately and personally, because our product sits in the middle of their working day. When Atlas is down, someone cannot send the contract, cannot find the task, cannot do the thing they came to do. The trust that takes years to build erodes in the minutes an outage lasts.
That is the case for treating reliability as a feature, and it is not a soft one. Reliability is the feature that makes every other feature usable. A brilliant new capability that is unavailable two afternoons a month is not a brilliant capability, it is a liability with good marketing. The engineering practices that protect uptime are not a cost center sitting beside the product. They are part of the product, arguably the most important part, because they are the difference between software people depend on and software people tolerate.
If there is one thing I would tell another engineering leader, it is this. Stop asking your team to care more about reliability and start giving it the same structure you give everything you take seriously. An owner. A roadmap. A budget. A scoreboard the decision-makers can see. The moment reliability competes for resources on equal terms instead of begging for scraps after the fact, it stops being the thing that breaks and starts being the thing you can count on. That is what it means to treat it like a feature, and it is the best engineering decision we have made.