How to write a hypothesis that can be proved wrong

The simplest structure for a hypothesis is if, then, because. If we make this change, then this behaviour will move, because we believe this mechanism is driving it. It makes you name the treatment, the effect and the reasoning, and that alone kills a lot of bad ideas early.

The trouble is that the format invites you to write it backwards. It starts with the treatment, so it’s easy to pick a change first, usually one that’s quick to build, and come up with a plausible because afterwards. Check your own backlog and you’ll see it: lots of tests that move something on the page, edit a heading or change a colour. Nobody observed a behaviour that called for them. The visual editor made them easy, and that’s where the idea came from. The fix is a rule: no hypothesis reaches the backlog without some prior evidence attached, whether that’s an observation, a piece of research or a pattern in support tickets. If you can’t name the mechanism, you don’t have a hypothesis yet.

Good hypotheses come from research, past learnings and plain observation. Take a senior person’s hypothesis as seriously as anyone else’s. If a founder is convinced the nav is losing sales, arguing with them gets you nowhere. Go and find or build the evidence instead, and an opinion nobody can settle becomes a claim you can test.

The because is the part most often left out, and it matters most. Stating your outcome metric before the test runs stops you picking your favourite number afterwards. The because goes further and commits you to a claim about behaviour: why the number will move, as well as that it will. Skip it and you’re left with a result you can’t learn from.

Watch for a hypothesis that slowly grows to cover several changes at once. A full page redesign might tell you the new page beats the old one, but it won’t tell you why, because every change in it is a separate variable you can’t pull apart afterwards. Split a big idea into smaller tests with one mechanism each and you get answers you can reuse.

The most common bad hypothesis starts from aesthetic preference. Someone likes the look of a carousel and works backward to justify it, when the idea came from seeing one on a competitor’s homepage.

Worse still is a hypothesis that can’t be wrong. “This will improve the user experience” sounds reasonable, but no result can contradict it. If conversion rises, the experience improved. If it falls, something else masked it. If it’s flat, run it longer. A test of that can only confirm what you already believed, so it can’t teach you anything.

A good hypothesis is one that can lose. It has a real mechanism, real evidence, and a result that could go either way, so even a losing test teaches you something. Afterwards, if you can point at your because and say plainly whether it held up, you wrote it well, whatever the number did.

Put it to work

Take your next test idea and write down, in one sentence, what result would prove the hypothesis wrong. If you cannot answer that, it is not ready to test yet.

Your answer

Show the worked example

There is no right answer. Write yours first, then compare.