Skip to content
6 min read

A/B testing website changes: validate hypotheses honestly

A/B testing website changes is the only way to learn what genuinely brings in more leads, rather than what the director or the designer happens to like. But a badly run test is worse than none, because it gives you confidence in a false conclusion. Below: what to test, how much traffic you need, how to write a hypothesis, and what to do when your site has few visitors, as is the case for most small businesses in Uzbekistan.

A/B testing website elements: what to test first

In order of impact: the offer above the fold, the call to action, block structure and order, form length, and how the price is presented. Button colour comes last on the list, because it rarely decides whether a person leaves a request.

Start with what every visitor sees. Testing a block that only five percent of visitors scroll to is pointless: it has almost no audience. Look at the scroll map and pick the most visible part of the page.

Typical first hypotheses for a landing page: a headline that states a concrete benefit instead of a generic slogan; a button that says ‘Get a quote’ instead of ‘Submit’; a two-field form (name and phone) instead of five fields; a ‘from’ price on the first screen instead of a hidden one; Telegram as an extra contact option next to the phone number, since part of the audience finds it easier to message than to call.

How much traffic you need

The core problem for small businesses: 200 visits a week cannot produce a statistically significant result. A two-lead difference is noise, not improvement.

A rough calculation. Say your conversion rate is 3%. To reliably detect a rise to 3.6% (one fifth higher) you need about 14,000 visitors per variant. To detect a rise to 4.5% you need roughly 2,500 per variant. These are guideline figures at the standard 95% confidence and 80% power. At 200 visits a week even the second case takes about 25 weeks. Use any online sample size calculator for your exact numbers.

The conclusion: with low traffic, test large changes — a fundamentally different offer, not a rephrasing — because only large effects are visible in small samples.

How to write a hypothesis

A good hypothesis fits in one sentence: ‘If we replace X with Y, the lead conversion rate will rise, because Z.’ Without the ‘because’ it is just a wish to change the design.

Before launching, write down the primary metric (a submitted request, not a button click), the minimum sample size, the duration and the stopping rule. Do not change the metric during the test.

The source of hypotheses is data: session recordings, questions in chats and requests, calls to the sales team. If customers keep asking ‘how much does it cost?’, the problem is how the price is shown, not the colour of the button.

Rules for a valid test

One hypothesis at a time, otherwise you will not know what worked. Split traffic randomly and equally, and run both variants simultaneously. The approach of ‘the old version in January, the new one in February’ fails because seasonality and ad campaigns distort the result.

Run the test in whole weeks to capture every weekday, since Monday and Saturday behaviour differ. Two weeks is the usual minimum, provided the calculated sample has been reached by then.

Do not stop the test the moment your preferred variant leads. That is the most common mistake: the sample is not yet collected, but the decision is already made. Other typical errors: changing a variant mid-test, counting clicks instead of requests, mixing traffic sources of different quality, and never looking at mobile visitors separately.

What to do with low traffic

Use qualitative methods instead of A/B: session recordings, heatmaps (Yandex Metrica with Webvisor and Microsoft Clarity are free), and interviews with five real customers about why they chose you. This produces hypotheses faster and cheaper.

It often turns out the problem is not the button but that the customer did not understand the service or could not find the price. Apply such findings immediately, without a test: an obvious mistake does not need statistics.

A one-week checklist: 1) watch 20 session recordings from phones; 2) find the field where people abandon the form; 3) check load speed on a mobile connection; 4) ask managers which questions customers ask most often; 5) fix the top three problems. An A/B test is the next step, once traffic grows.

FAQ

How long should an A/B test run? Until the sample calculated in advance is reached, and not less than one or two full weeks; the exact time depends on traffic and conversion rate.

Can I test several changes at once? You can, but you will only learn whether the whole bundle worked, not which part. For clean conclusions, test one hypothesis at a time.

Which tool should I choose? Google Optimize was shut down in 2023; today there are dedicated testing platforms, and a split can also be implemented on the website side. What matters is random assignment of visitors and correct counting of requests.

What if the result is borderline? Treat the test as inconclusive and do not roll out the ‘winner’ on a hunch: either extend the test to the calculated sample or try a larger change.

Where to start

If you do not have a site or landing page yet, A/B tests begin not with buttons but with a solid base: a clear offer, a fast mobile version, and analytics with goals configured. The 4 Digital studio builds turnkey landing pages starting from 1,399,000 UZS; the final price depends on the number of blocks, the amount of copy, the design and integrations such as a Telegram bot or CRM. Start with one page, collect data, then check hypotheses one by one.

Need a website or ads? Let’s discuss your project.

Turnkey Landing Page · от 1 399000 сум

We reply within 15 minutes during business hours. No spam.