CRO beyond A/B tests — what big teams skip
A/B testing is the obvious tool, but on modest traffic it's slow. How to diagnose conversion problems first and test only what's genuinely uncertain.
A/B testing is the tool everyone reaches for when a page doesn't convert. It's also the tool that works worst for most small and mid-sized sites. A test needs enough visitors and enough conversions to tell a real difference from random noise, and with modest traffic a single test can take months to reach a reliable answer. Meanwhile the team waits. Or, worse, it stops the test early and ships whichever variant happened to be ahead that week.
Product teams with huge audiences can afford to test everything. Everyone else needs a different order of work: diagnose first, fix the obvious, and test only what's genuinely uncertain.
Check whether A/B testing even fits
Before planning a test, put your numbers into a sample size calculator. Most testing tools include one, and there are free ones online. You'll need the page's current conversion rate, the smallest improvement you'd care about, and weekly traffic. If the answer is longer than you're willing to wait, a test is the wrong tool for that decision. Smaller sites usually find that only large, bold changes can be tested in reasonable time. Subtle tweaks, like button color, can't.
Diagnostic tools that work on small samples
- Session recordings and heat maps. Tools such as Microsoft Clarity or Hotjar show where people click, how far they scroll and where they hesitate. A few dozen recordings of visitors who abandoned a form often show the problem plainly: a field that rejects valid input, a button pushed below the fold on mobile, a price that only appears at the last step.
- Five-second tests. Show the page to people for five seconds, then ask what they remember and what the company does. Services like Lyssna (formerly UsabilityHub) run these with recruited participants. If people can't say what you sell, no amount of button testing will fix it.
- One-question surveys. A short, polite question on the pricing or checkout page: "What's stopping you from continuing today?" The answers are qualitative, but recurring themes show up quickly.
- Talking to sales and support. The people who handle inquiries know which questions the site fails to answer. It's the cheapest research you'll ever do.
- Funnel reports in analytics. Define the steps from landing to inquiry and find the biggest drop-off. Start there, not with the home page hero.
The order that saves time
- Find the drop-off. Which step loses the most people?
- Watch it happen. Recordings and surveys on that step.
- Name the cause. Friction (forms, speed, bugs), a trust gap (no reviews, unclear terms), or a mismatch, where the page doesn't answer the question that brought people there.
- Fix clear problems directly. A broken field on mobile doesn't need a test. It needs a fix.
- Test what's uncertain. Two genuinely different value propositions, two pricing layouts. Big, meaningful differences.
Skipping diagnosis means testing random ideas. Each test costs weeks, and most random ideas don't move anything.
Common mistakes
- Stopping tests early because one variant looks like it's winning. Early leads often vanish.
- Testing tiny changes on low traffic. The effect is too small to detect.
- Running overlapping tests on the same audience and then not knowing which change caused what.
- Ignoring speed. A slow page drags every variant down equally and hides real differences.
- Optimizing a metric that doesn't matter. More clicks on "Learn more" isn't the same as more inquiries.
When A/B testing is the right tool
Use it on high-traffic pages, for decisions where you honestly don't know the answer and the stakes are high: pricing presentation, the signup flow, the main offer. Use the diagnostic tools above for everything else. For changes that are usually safe to make without a test, see CRO heuristics.
If you'd like an outside view of where your site loses people, our website audit covers analytics, funnel drop-off, speed and usability, and we can rebuild the problem page as a focused landing page.
Want to find out where visitors drop off? Reach us on Telegram or MAX.
Got a project on this topic? Tell us the task — we'll come back with an estimate and a schedule.
Message on Telegram → Message on MAX