What segmentation of results measures
An overall result is an average of everyone who took part, and averages hide disagreement. Segmentation splits the same data by something you already know about the visitor — device, traffic source, country, new against returning, logged in against anonymous — and asks whether the effect was the same in each group.
Sometimes the answer is dull and reassuring: the change helped everyone by roughly the same amount, so the average tells the whole story. Sometimes it is not. A shorter form can lift mobile conversions while doing nothing on desktop, where the longer version was never the obstacle. A reworked page can win on search traffic and lose on paid traffic, because those visitors arrive with different expectations.
The important discipline is that segmentation reuses data collected for one comparison. Each split makes the groups smaller and the numbers noisier, so a segment result is always weaker evidence than the headline it came from.
Why segmentation of results matters
It stops a flat overall result being thrown away. A test that looks like nothing is quite often a clear win in one group cancelled by a clear loss in another, and the honest response is to serve the two groups differently rather than to conclude the idea failed.
In Nepal and similar markets the device split does most of the work, because traffic skews heavily towards mobile phones on variable connections. A desktop-flattering result that quietly fails on mobile is failing for most of the audience. Channel splits matter for the same reason: visitors from a conversion funnel that starts with an ad have already been sold something, and they read a page differently from someone arriving cold from search.
Common mistakes with segmentation of results
The big one is cutting the data until something looks impressive. Slice by enough dimensions and some segment will always appear to be a winner purely by chance, and reporting that slice as the finding is a way of manufacturing results rather than discovering them.
The second is acting on segments too small to mean anything. When a split leaves only a scattering of conversions on each side, the difference between the groups is mostly noise, and the range around it will be wide enough to include no effect at all.
The third is treating a segment discovery as proven. A split you did not plan is a hypothesis, not a conclusion. It earns a follow-up test aimed at that audience; it does not earn an immediate site-wide decision.
What to do about it
Decide before launch which two or three splits you will look at and why, and write them down. Device and traffic source are almost always worth it; long lists of demographic cuts rarely are. Naming them in advance is what separates analysis from fishing.
When you do find a difference, check the size of the groups and the range around each figure before you believe it, then ask whether the pattern makes sense as a story about behaviour. A mobile-only lift on a change that removes typing is plausible. A lift confined to one browser usually means a rendering bug worth investigating rather than an audience insight. Feed real differences back as the next test, and keep the site-wide decision anchored to the overall result.
