Analytics and Tracking

Hypothesis

Also called test hypothesis

A written prediction naming the change, the measure expected to move and the reasoning, agreed before a test begins.

Quick facts: Hypothesis

Category
Analytics and Tracking
Also called
test hypothesis
Level
Beginner
Affects
Test design, what counts as success, learning from losses
Where to see it
A written test log or spreadsheet, testing platforms such as VWO or Optimizely
In this article4
  1. What a hypothesis contains
  2. Why a hypothesis matters
  3. Where hypotheses go wrong
  4. How to write a good one

What a hypothesis contains

A usable hypothesis has four parts, written down before anything is built. The observation: what you have seen in the data or in customer behaviour that made you suspect a problem. The change: precisely what will be different in the variant. The expected effect: which single measure should move, and in which direction. The reasoning: why you believe the change causes that effect.

Written out, it reads plainly. Visitors are abandoning the enquiry form at the phone number field; removing that field will raise completed enquiries, because the field asks for information people are reluctant to give before they know the price.

The reasoning is the part most teams skip, and it is the part that earns its keep. It is what makes the test transferable — if it holds, you have learned something about your customers that applies to other pages, not just that one page won.

Why a hypothesis matters

It forces the decision about what counts as success to happen before anyone can see the result. Without that, a test that fails on its intended measure gets rescued by whichever secondary number happened to rise, and the team congratulates itself on nothing.

It also makes a losing test valuable. If the reasoning was clear, a loss tells you the belief behind it was wrong, which is genuine information. A test with no stated reasoning that loses tells you only that one page did worse than another, and nothing you can carry forward.

Where hypotheses go wrong

The commonest fault is the untestable wish: making the page better will improve conversions. Nothing in that sentence can be measured or disproved. The second is a hypothesis written to justify a change somebody had already decided to make, which turns the test into a formality.

Then there is the hypothesis with too many expected effects. If four measures are named as success, one of them will move, and you will not know which change caused it. Naming several changes in a single prediction has the same problem: when the result arrives, you cannot say which part did the work.

How to write a good one

Start from evidence rather than opinion. Session recordings, form drop-off, search queries, support emails and the questions clients ask on the phone are all better sources than a meeting. Then name one change, one primary measure and one reason.

Decide the required sample size and the run length in the same document, so the finishing line is set before the race. Keep a written record of every hypothesis and its outcome, including the losses, because the pattern across them teaches more than any single test. In practice this record becomes the backbone of a conversion rate optimisation programme, and it stops the same idea being retested every year.

Do and do not

Do

  • Name one change, one measure and one reason
  • Base it on evidence rather than a meeting opinion
  • Record the outcome, including tests that lost

Do not

  • Write a wish that cannot be disproved
  • List several success measures in one prediction
  • Write it to justify a decision already made

Questions people ask about this

What does a good hypothesis look like?

It names one change, one measure and one reason, in language anyone can check. For example: visitors leave the enquiry form at the budget question, so removing that field will increase completed enquiries, because people will not state a budget before they know your prices. Every part of that can be tested and disproved.

Do I need a hypothesis for every test?

For anything you intend to learn from, yes. Without one, you cannot say in advance what success means, so the result gets interpreted after the fact and usually in whichever direction suits the person reading it. Small technical fixes that are obviously correct do not need a test at all, let alone a hypothesis.

What if the test disproves my hypothesis?

That is a useful outcome, provided the reasoning was written down. A loss tells you the belief behind the change was wrong, which narrows the field for the next idea and stops the same assumption being retested later. Record it alongside the winners; a log of disproved beliefs is worth keeping.

Related terms

Found this useful?

Share it, or ask an AI to summarise it

Back to the glossary

Knowing the term is the easy part

Applying it to your own site and budget is the work. Book a call and I will tell you what actually applies to you.