A few years ago I inherited a service with ninety-four percent test coverage and a reputation for being terrifying to touch. Both facts were true at once, and they were related. The tests were not protecting the codebase, they were guarding it. Half of them asserted that a function called the three other functions it was implemented to call, so any refactor, even one that preserved behavior perfectly, broke a dozen tests that knew too much about the internals. The team had learned to fear changes, not because the code was fragile, but because the test suite was. That suite had high coverage and paid no rent.
This is the trap with test coverage as a target. It measures how many lines execute while your tests run, which is not the same as how many bugs your tests would catch, and certainly not the same as how confidently your team can ship. We stopped treating coverage as the goal and started asking a sharper question of every test we wrote: does this test earn its keep? A test costs something to write and far more to maintain over its life. If it does not prevent a real bug or give us real confidence to change code, it is a liability wearing a green checkmark.
Coverage is a map, not the territory
I am not against measuring coverage. It is a useful map of which parts of the system your tests never visit, and a sudden drop in coverage on a pull request is a fair thing to flag. The mistake is treating the number itself as the objective. The moment a coverage percentage becomes a target, engineers optimize for the number, and you get tests that touch lines without asserting anything meaningful about them. A test that runs a function and checks nothing about the result still moves the coverage needle. It also catches nothing, ever.
So we inverted the framing. Instead of "what is our coverage," we ask "what would have to break for a customer to notice, and is that path tested." A payments flow with eighty percent coverage where the twenty percent is the refund edge cases is in far worse shape than a logging utility at forty percent. Coverage averages hide exactly the risk you care about. The map is fine. Just do not confuse it for knowing where the cliffs are.
The tests that pay rent
When we audited which tests had actually caught regressions over a year, a clear pattern emerged. The valuable ones shared traits, and so did the worthless ones. The tests worth keeping tend to look like this:
- They test behavior, not implementation. They assert what the code does for a caller, not how it does it internally, so they survive refactors and only fail when behavior actually changes.
- They cover the paths that hurt. Money, auth, data loss, anything irreversible. These get tested thoroughly and defensively, because the cost of a bug there is enormous.
- They fail for one clear reason. When a good test goes red, it points at exactly what broke, so a failure is information rather than a scavenger hunt.
The flip side is just as clear. The tests we deleted without regret were the ones tightly coupled to internals, the ones that mocked so much that they only verified the mocks, and the ones so flaky that the team had learned to re-run the suite until it passed. A flaky test is worse than no test. It trains people to ignore red, and a team that ignores red will eventually ignore the one failure that mattered.
Test at the right altitude
Most software quality debates collapse into the testing pyramid, and most teams get the proportions wrong in practice even when they agree with it in theory. The instinct is to write unit tests for everything because they are fast and easy. But a unit test on a thin wrapper that just forwards a call proves almost nothing, while it still has to be maintained forever. The bugs that actually reach customers usually live in the seams between units, in the integration, where a unit test by definition cannot see them.
Our rule of thumb is to test at the highest altitude that still gives a clear, fast failure. A focused integration test that exercises a real request through real layers to a real test database catches a whole class of bugs that a hundred mocked unit tests would miss, and it does so while testing behavior rather than structure. We reserve heavy unit testing for genuinely complex logic: pricing rules, scheduling, parsing, the places where the algorithm itself is hard. For the plumbing that connects those pieces, integration is where the rent gets paid.
It is easy to have thorough unit coverage on every component of a flow and still ship a bug, because two correctly-tested pieces disagree about a date format at the boundary between them. Every unit test is green. One integration test through the actual path catches what a hundred mocked unit tests cannot, and for something like a contract-signing flow in Atlas, that is exactly the kind of test worth writing first.
A practical filter for what to write
The framework we settled on is a short set of questions we ask before writing any test. What real failure does this prevent. Will it still pass after a reasonable refactor that keeps behavior intact. When it fails, will the cause be obvious. If a test cannot answer those three clearly, it probably is not worth writing, and an existing test that fails them is probably worth deleting. Deleting tests makes engineers nervous, and it should be done thoughtfully, but a suite you trust at eighty percent coverage is worth more than a suite you fear at ninety five.
This connects to a broader engineering value we hold, which is that the test suite is part of the product's quality, not a chore tacked on after the real work. The goal of testing is not a number in a report. It is the confidence to change code quickly without breaking what customers rely on. Software quality is what lets a team stay fast as it grows, and a suite full of tests that pay their rent is what makes that speed safe. Stop counting lines. Start asking which tests would actually save you on a bad day, and write those.