How A/B testing works
You take one page, one email or one ad, change a single thing, and show the original and the variant to comparable slices of the same live audience at the same moment. The tool decides at random which visitor sees which, so the only systematic difference between the groups is the change you made. Any difference that then appears in the conversion rate can be credited to that change rather than to the day of the week, a campaign that started midway or a different mix of traffic sources.
Randomisation and simultaneity are what make it a test rather than a comparison. Running the old page one month and the new page the next is not an A/B test; it is a before-and-after guess with the season, the weather and every other campaign left in.
Why A/B testing matters
Opinions about layout, wording and colour are cheap and endless. A test replaces the argument with evidence from the people who actually buy, which is useful when the loudest opinion in the room belongs to whoever is paying for the site.
It also protects you from your own good ideas. Plenty of changes that look like obvious improvements — a longer form to qualify leads, a bolder discount, a video in place of a headline — turn out to do nothing, or to do harm. A test catches that before it becomes permanent.
Common mistakes with A/B testing
The biggest is stopping the moment the numbers look good. Early results swing wildly, and a variant that appears ahead in the first days is often level or behind by the end. Statistical significance exists to tell you when a gap is larger than ordinary noise, and ignoring it turns testing into an expensive way of confirming what you already believed.
The second is testing on traffic too thin for the effect you hope to find. A site with a few conversions a week will not resolve a small difference in any sensible time frame, and it is more honest to fix the obvious problems than to run a test that cannot conclude. The third is changing several things at once, which may tell you the new page is better without telling you which part did the work.
How to act on it
Test where the traffic and the money already are: the main landing page, the checkout, the enquiry form. Write down what you expect to happen and why before you start, because a prediction you have committed to is much harder to rationalise afterwards. Run whole weeks so weekday and weekend behaviour are both included, and leave the winner in place long enough to confirm it holds.
Where volume is too low to test at all — common for specialist business-to-business services and for smaller markets such as Nepal — use the parts of conversion rate optimisation that need no statistics: session recordings, watching a few real customers attempt the task, and removing the obstacles you can see with your own eyes.