Deciding which test to run first

You’ve got a backlog of evidence-backed ideas, which is good, but you still need a plan. You can’t run them all at once, and the order matters, because traffic and patience are finite. The question is how you choose, and in most teams the default answer is that the most senior person in the room does. That’s how a programme set up to replace opinion ends up bringing it back in at the prioritisation stage.

A prioritisation framework makes the ranking transparent and consistent. The common ones all work the same way. You score each idea on a few dimensions, usually impact, confidence and ease, and the total gives you an order. ICE is impact, confidence, ease. PIE is potential, importance, ease. RICE adds reach and effort. Which model you pick matters less than using one, because a consistent score lets an idea’s rank depend on the idea rather than on who argued for it. There’s nothing clever about it. It just stops the meeting being a popularity contest.

Do the scoring before the arguing, and ideally on your own. Score the ideas in advance and the meeting becomes a discussion of the numbers rather than a contest of wills. When someone disagrees with a rank, the useful question is “where do we disagree, on impact or on confidence?”, and that’s a conversation you can have, because it’s about the evidence. A framework turns “I just think it’ll work” into “what makes you rate the impact a five?”, and that shift is most of the value.

Watch out for the framework used as a rubber stamp. You decide what you want to build first, usually because someone senior is attached to it, then score the ideas so the favourite comes out on top. The framework gives the decision a scientific-looking number, but the number was chosen to flatter the conclusion. You can spot it because the scoring only ever happens once, after the decision, and nobody can quite remember how the ease got to be a five. A framework won’t catch anyone fudging the numbers, and it makes very convincing cover for it.

So score fairly, before the meeting, and let the order be whatever it is, including when that puts the boss’s idea fourth. Otherwise the backlog can look disciplined while still being the loudest person’s priorities written in a table. A programme that can say “we ranked it and this is what the numbers said” gets to keep testing, because people can see the reasoning behind each decision, and nobody has to win an argument to live with it.

Put it to work

Take your top five test ideas and score them on impact, confidence and ease, one to five each, on your own, before anyone else argues. Compare the order to your gut.

Your answer

Show the worked example

There is no right answer. Write yours first, then compare.

Sources

1. CXL, Prioritisation Frameworks