You can't optimise a page you don't understand

Most teams start a conversion project with an idea, and it usually sounds confident: change the button, move the form, rewrite the headline. What they rarely have is a reason. The idea felt right in a meeting, or a competitor does it, or someone senior liked the look of it. That’s more of a hope than a plan, and the point of research is to replace the hope with something you can act on.

A test can’t tell you what to build. It tells you whether a change you already made moved a number. If the change was aimed at the wrong problem, the test gives you a clean, honest and completely useless result. So the job of research is to find the problem before you spend a test on a fix. It answers the questions a test can’t: who uses this, what are they trying to do, and where does it go wrong for them? Until you know that, every change is a shot in the dark, even if it’s fired with good statistical hygiene.

The methods split into two families, and confusing them is where most research money disappears. Qualitative research explores the why. You watch or talk to a handful of people and learn the reasons behind what they do. It’s small and deep, and it’s where the insight comes from. Quantitative research measures how many: analytics, surveys, controlled tests. It’s large and shallow, and it sizes the problem or checks the fix. A funnel will show you that half your visitors leave at the payment step, but it won’t tell you why they leave. That’s why the two aren’t rivals. They answer different halves of the same question.

Good research starts with a question matched to a method. “Do people understand our delivery options?” is a qualitative question, a usability test. “How many people who reach the basket go on to finish?” is a quantitative one, a funnel read. “Would you buy this?” is neither, because it asks someone to predict their own future behaviour, and people can’t. Plenty of bad research goes wrong by asking a method a question it was never built to answer.

Be wary of research theatre, where the study exists to justify a decision that’s already been made and the findings are a formality on the way to shipping what someone wanted all along. You can usually spot it because the method was chosen after the conclusion. Research is only worth paying for if you’re prepared for it to prove you wrong, so keep the order: pick the question, then the method, then run it, and let the answer be whatever it is, including the one you’d rather not hear.

Put it to work

Before your next change, write down the problem it solves and the evidence you have that the problem is real. If the second column is empty, that is your next research task.

Your answer

Show the worked example

There is no right answer. Write yours first, then compare.

Sources

1. Erika Hall, Just Enough Research
2. NN/g, When to Use Which User-Experience Research Methods