Deploy is not release: split them on purpose

Shipping without a releaseOpinionOct 26, 20265 min readMiniwall team

Part of the series: Ways to change a live app without shipping a release

Deploying means the code is running on your servers or sitting inside a build. Releasing means a user can see what it does. Most teams treat these as one event, because for a long time they were. Our view is that they're two decisions, usually made by different people, and you get a calmer week if you make them separately on purpose.

One change, two decisions. The first lane is about code, the second is about people. Shape only, not data.

That picture is the whole idea. The rest of this post is about why it's worth the trouble, what it costs, and the cases where we'd skip it.

What goes wrong when they're one thing

When deploy and release are the same button, every deploy is a gamble taken with an audience watching. The code might be fine, but you find out only when users hit it. If it's not fine, your only way back is another deploy, which is the slow, nervous step you were trying to avoid.

It also ties two clocks together. Engineers want to merge small and often, because small changes are easier to review and to undo. Whoever owns the feature wants to launch on a Tuesday, after the pricing page is updated and support has been told. If merging means launching, one of those people loses. Usually the merges get bigger and rarer, which makes every deploy riskier, which is the opposite of what anyone wanted.

What splitting them gives you

Three things, in the order we'd care about them.

Merging stops being scary. Unfinished work can sit in the main branch, switched off, instead of living for three weeks on a branch that drifts away from everything else. The conflicts you'd have resolved in one painful afternoon get resolved a little at a time.

You can test in production without a launch. The new path runs for you and your team before anyone else. People call this a dark launch when the code is running but nobody outside sees the result. We haven't measured how much this catches. Our guess is that it finds the boring failures (a missing environment variable, a query that is slow on real data) that a staging environment tends to miss.

Release becomes a dial, not a door. Ten percent, then a day of watching, then everyone. If something looks wrong you turn the dial back, and nobody needs to rebuild anything. The decision to turn it is a business decision, so it can sit with whoever owns the outcome.

What it costs

This is the part people skip when they recommend flags, so let's be plain about it.

Every flag is an extra path through your code. While it exists, both paths ship, and both need to work. Ten independent on-off flags give 1,024 combinations. Most can never happen together, but the code doesn't know that and nobody has tested them.

Dead flags pile up. Uber's engineers built a tool to delete the leftovers. Their README says the code for a flag often stays "even after the purpose of the flag is accomplished", and lists the damage: "Presence of unused code in the source as well as the binary", and "Stale flags can also cause bugs."^[2] When a company writes a tool just for the cleanup, that tells you the cleanup is real work and doesn't happen by itself.

The off state needs a test. It's easy to test the new path and forget the old one. Do the opposite: the path that runs when the flag is off is the one that protects you, so it's the one to keep honest.

You need a safe answer when the flag can't be read. The OpenFeature specification puts it in one line: "Flag evaluation calls must always return the default value in the event of abnormal execution."^[1] Decide what the default is before you write the check. For anything new, the default is off.

None of this argues against splitting. It argues for doing it deliberately, with a price tag in mind.

How we'd keep the cost down

These are habits we'd suggest, not a standard and not drawn from any study.

Write the removal date when you create the flag, and put a name next to it. "Later" is not a date.
Keep release flags short. When the rollout reaches everyone, deleting the flag and the losing path is the last step of the project, not a task for someday.
Prefer a few large launches behind one flag over many small flags that depend on each other.
Keep the check at one place in the code, near the edge, instead of scattering it through ten files.

Who decides the release

Once the two are split, a new question appears: who turns the dial? Deploying is an engineering decision. It's about whether the code is safe to run. Releasing is about whether users should see it now, and that depends on things code can't tell you: the campaign calendar, a support backlog, a price change that has to land first.

In many teams we can imagine, that second decision still waits in a developer's queue because the only switch lives in a config file. We'd rather the person who owns the launch can make it, within limits the code enforces, and with a log of who did what. If the changes you keep waiting on are screens and copy, the options for changing a live app without a release start from the lightest, and what server-driven UI is covers the one where the layout itself comes from the server.

When we'd skip it

If you deploy several times a day, the team is small, and a rollback is one click, a flag may be more machinery than the problem deserves. Just deploy. The same goes for changes that can't be hidden: a database change that breaks the old code, or a screen that depends on a permission you haven't been granted yet. A flag in front of those gives you a feeling of safety and little else.

And a flag is not a security control. If something must not be seen, enforce that on the server.

Our rule of thumb: split deploy from release for anything a user would notice, or anything you'd hesitate to roll back by redeploying. For the rest, the simplest process wins. We haven't tested any of this against your team, so treat it as a place to start arguing from.