Championing experimentation

You never have to blame the weather againWhat a non-technical stakeholder needs is a plain explanation of the method, realistic expectations on pace and win rate, and a practitioner they can trust.Lesson 1 · 7 minReporting wins and losses the same wayReport a win and a loss in the same words, give the range behind the number, and don't cherry-pick, so the next result still gets believed.Lesson 2 · 4 minWhat sits behind the common objections to testingThe common objections, we already know, just ship it, tests take too long, usually share a root.Lesson 3 · 4 minWhen a test is the wrong toolTesting is the wrong tool often enough that knowing when to skip it is part of the job.Lesson 4 · 4 minHow to brief a testing team so the work answers your questionA good experimentation brief sets out the business goal, the evidence, the constraints and who decides what, and leaves the design of the test to the specialists.Lesson 5 · 5 minChecking a results deck before you believe itA commissioner's checklist for any results presentation, from whether the metric was fixed in advance to why annualised revenue figures run high.Lesson 6 · 5 minWhat to ask an agency or consultant before you hire themBefore you hire an experimentation agency or consultant, ask how they find ideas, how they handle the statistics, how they report losses and who keeps the learnings.Lesson 7 · 5 minHow to make the business case for an experimentation programmeA credible case for experimentation costs the whole programme, people, tools and development time, and values it as better decisions and losses avoided as well as wins.Lesson 8 · 5 minNext track: Embedding the disciplineBuild a habit of testing that the whole organisation owns, so it outlasts any one person.Track 6