Engineering

Scaling a system is mostly about deleting things.

When people picture scaling, they imagine adding more servers. In practice, most of our hardest scaling wins came from removing work that never needed to happen in the first place: extra queries, redundant background jobs, and features nobody actually used. Subtraction scales far better than addition.

The picture most people carry around for scaling is a wall of servers and a graph going up and to the right. More load, more machines. That picture has caused more wasted engineering effort than almost any other idea I can name. In my experience the largest scaling wins do not come from adding capacity. They come from removing work the system was doing that it never needed to do. When you delete that work, the remaining system gets faster, cheaper, and simpler all at once, and you did not buy a single extra instance to make it happen.

I want to be precise about what I mean, because deleting things sounds reckless and it is not. It is the most disciplined form of scaling there is. Adding capacity hides problems behind money. Deleting work forces you to understand the problem well enough to know what is safe to remove, and that understanding is the actual asset. A team that scales by subtraction ends up with a system architecture it comprehends. A team that scales by addition ends up with a system that is large, expensive, and increasingly mysterious to the people who own it.

Most load is self-inflicted

When we profile a service that is struggling, the load is almost never the load we imagined. It is rarely the headline feature buckling under genuine user demand. It is usually a quiet pile of work the system is doing to itself. A query that runs on every request when it could run once and be cached. A background job that re-processes the same records on a five minute timer because someone, years ago, was not sure the first run had taken. An endpoint that returns forty fields when the caller reads two. A retry policy that turns one failed request into eight. Individually, each is small. In aggregate they are most of your bill and most of your latency.

The N plus one query is the canonical example, and it is canonical because it is everywhere. A page loads a list of fifty items and then, almost invisibly, fires fifty more queries to decorate them. Nobody designed that. It accreted, one reasonable-looking line at a time. The fix is not a bigger database. The fix is to delete forty-nine queries by fetching what you need in one. We have cut response times by more than half on important paths doing nothing more glamorous than finding and removing that pattern, and the database load dropped as a side effect we did not even have to plan for.

The cheapest work is the work that never runs

There is a hierarchy of optimization that I drill into every engineer who joins. The fastest code is the code you delete. The next fastest is the code that does not run because you skipped it. Only after both of those have been exhausted does it make sense to make the code that does run go faster. Most engineers reach straight for the third option, because making something fast feels like real engineering, while removing something feels like admitting it should not have been there. The order is backwards, and it costs teams enormous amounts of effort and money.

Caching is the obvious instance of skipping work, but caching is a tool that punishes you if you do not understand the work first. People reach for a cache to paper over a computation they never examined, and now they own a fast wrong answer and a hard invalidation problem. The right sequence is to understand the work, delete what you can, and only then cache what genuinely must be computed and can safely be reused. Performance work done in that order compounds. Done in the wrong order it just adds a second system on top of the one you already did not understand.

Example: we once had a dashboard that recomputed an expensive aggregate on every page view. The instinct in the room was to cache the result. Instead we asked who actually looked at it and how fresh it needed to be. The answer was a handful of people, a few times a day, for whom hourly freshness was fine. We deleted the per-request computation entirely and moved it to a job that runs hourly. The load on that path effectively went to zero, and the cache we almost built would have been more complex than the problem deserved.

There is a deeper reason addition is so seductive and so dangerous. Adding capacity gives you an immediate, legible win. The graph that was red turns green, the on-call pages stop, and everyone can point to the new instances as proof that the problem was handled. Deletion gives you the same outcome with none of the visible heroics, and worse, it requires you to first admit that the system was wasteful, which feels like criticizing whoever built it, often yourself. So teams reach for the option that looks like progress and feels like nobody's fault, and they pay for that comfort every month in their infrastructure bill. The discipline of subtraction is partly technical and partly a willingness to be unglamorous about what good engineering actually looks like.

Concurrency does not fix wasted work

A close cousin of buying more servers is throwing more parallelism at a slow path. If a job is slow, run more workers. If a request is slow, fan it out. Sometimes that is the right answer, when the work is genuinely necessary and genuinely parallel. But far more often, parallelism is just a way to do wasteful work faster, and you have now spent engineering effort building a concurrency mechanism to accelerate computation that should not have been happening at all. You also bought yourself a new class of problems: coordination, contention, partial failures, and the subtle bugs that only appear under real load. A piece of wasted work is cheapest to fix by deleting it, not by parallelizing it into something harder to reason about.

The same logic applies to most clever performance machinery. Queues, sharding, read replicas, and elaborate caching layers are all real tools, and we use every one of them where the underlying work justifies it. But each is also a place to hide a problem you never understood, and each adds permanent complexity that every future engineer has to carry. Before we add any of them, the question is always the same: have we deleted everything we can first, so that the machinery we are about to build is wrapping necessary work rather than enshrining waste. More often than the room expects, the honest answer is that the machinery is not needed once the waste is gone.

Delete features, not just code

The hardest deletions are not in the code. They are in the product. Every feature you ship is a permanent tax on every future scaling effort, because it is one more thing that has to keep working, keep being tested, keep being reasoned about when load shifts. A feature that three customers use and nobody on the team remembers building will still wake someone at three in the morning, and it will still be the edge case that makes the clean refactor impossible.

So we treat feature removal as a normal engineering practice, not a failure. We instrument usage, we look honestly at what almost nobody touches, and we sunset it with proper notice. The relief is real and immediate. Every deleted feature deletes its tests, its edge cases, its corner of the data model, and its share of the cognitive load that slows every change you make near it. Subtraction at the product level scales the team, which is the kind of scaling people forget to count.

Measure before you cut, and after

None of this is license to delete on instinct. Deletion is dangerous exactly in proportion to how little you understand. The discipline that makes subtraction safe is measurement. We do not remove a query, a job, or a feature without first knowing what it costs and who depends on it, and we confirm afterward that the thing we expected to drop actually dropped. Profiling, tracing, and real usage data are what separate confident deletion from gambling.

This is where good observability earns its place in your engineering practices. You cannot delete what you cannot see, and most teams are flying blind on their own internal load. They know their user traffic and almost nothing about the work their system generates for itself. Invest in seeing that, and the deletions reveal themselves. The system will tell you exactly which work is wasteful if you bother to ask it.

We run our own products under this philosophy. Keeping Atlas fast as it has grown has been far more about removing internal work, collapsing redundant queries, retiring jobs that outlived their reason, sunsetting features that did not earn their keep, than about provisioning our way out of trouble. The result is a system you can hold in your head, which is the only kind that stays reliable as it grows.

So when someone tells me a system needs to scale, my first question is never how much capacity we should add. It is what work the system is doing that it should not be doing at all. Optimization that starts with addition treats the symptom. Optimization that starts with deletion treats the disease, and it leaves you with something smaller, faster, and finally understandable. Scale by subtraction, and you will be surprised how rarely you have to scale by addition at all.

F

Farhan

Farhan is the solo builder of wrxstack. He designs, writes, and ships Atlas and Portfolio on his own, and writes here about product, engineering, careers, and the craft of building software as one person.