A/B testing: validating hypotheses without fooling yourself
An A/B test is the only way to learn what genuinely works rather than what the director likes. But a badly run test is worse than none.
What to test first
In order of impact: the offer above the fold, the call to action, block order, form length, and pricing presentation. Button colour comes last.
Start with what every visitor sees. Testing a block only five percent scroll to is pointless.
How much traffic you need
The core problem for small businesses: 200 visits a week cannot produce a statistically significant result. A two-lead difference is noise.
With low traffic it is more honest to test large changes — a fundamentally different offer, not a rephrasing — because only large effects are visible in small samples.
Rules for a valid test
One hypothesis at a time, and run in whole weeks to capture every weekday, since Monday and Saturday behaviour differ.
Do not stop the test the moment your preferred variant leads. That is the most common mistake.
What to do with low traffic
Use qualitative methods instead: session recordings, heatmaps, and interviewing five real customers about why they chose you.
It often turns out the problem is not the button but that the customer did not understand the service or could not find pricing.