Here's a scenario that plays out across nearly every eCommerce and SaaS brand running a testing programme: the team sits down on Monday, someone suggests changing the CTA colour on the product page, someone else wants to test a new hero banner, and the most senior person in the room lobbies for a pricing page redesign. Three tests get queued. None of them are connected to a hypothesis. None have been scored against potential impact. And by the time results come in six weeks later, nobody can articulate what was learned — or what to do next.
This is the testing treadmill, and most organisations are stuck on it. Research from Speero (formerly CXL Institute) shows that 60–70% of testing capacity at the average company is spent on low-impact or inconclusive experiments. That's not just wasted dev hours. It's wasted opportunity cost — the revenue that a properly prioritised test could have generated in the same window.
The fix isn't more tools or more traffic. It's a scoring framework that forces discipline into what gets tested and in what order. At Converzion Catalyst, that framework is RICE.
What RICE Actually Is (And Why It Works for CRO)
RICE was originally developed by Intercom's product team to prioritise feature requests. It stands for Reach, Impact, Confidence, and Effort. The formula is straightforward:
RICE Score = (Reach × Impact × Confidence) / Effort
The reason it transfers so well to CRO is that conversion testing shares the same fundamental challenge as product development: too many ideas, not enough capacity, and a dangerous temptation to go with gut feeling instead of structured evaluation.
Let's break each component down as it applies to conversion rate optimisation specifically.
Reach
Reach measures how many users will be exposed to the test variant within a defined time period — typically per month or per test cycle. This isn't the same as total site traffic. It's the traffic to the specific page or flow being tested.
A homepage test on a site doing 200,000 monthly sessions might have a Reach of 200,000. A test on the returns policy page might have a Reach of 4,500. The difference in potential revenue impact between those two is enormous, yet teams routinely treat all tests as equal.
How to score it: Pull the monthly unique pageviews or session count for the page or flow in question from GA4. Use the raw number. Don't normalise it.
Impact
Impact estimates the magnitude of the expected conversion lift per user who sees the variant. This is where hypothesis quality matters. A button colour change might shift behaviour marginally. Restructuring the entire checkout flow could double conversions.
Converzion Catalyst uses a 1–5 scale:
- 1 — Minimal: Cosmetic change, minor copy tweak. Expected lift under 2%.
- 2 — Low: Single-element change with some supporting data. Expected lift 2–5%.
- 3 — Medium: Structural change to a key conversion element (CTA, form, layout). Expected lift 5–15%.
- 4 — High: Multi-element change addressing a known friction point with qualitative evidence. Expected lift 15–30%.
- 5 — Massive: Full flow redesign or removal of a major conversion blocker. Expected lift 30%+.
Confidence
Confidence is the honesty check. It forces the team to quantify how much supporting evidence exists for the hypothesis. A test idea backed by heatmap data, user recordings, survey feedback, and competitor analysis gets a higher Confidence score than one that came from a brainstorm.
Score as a percentage:
- 100%: Multiple data sources confirm the problem. Previous tests in the same area show directional results.
- 80%: Strong qualitative data (user recordings, heatmaps) plus some quantitative backing.
- 50%: One data source suggests the opportunity. Reasonable hypothesis but limited evidence.
- 30%: Based on best practice or competitor observation. No site-specific data.
- 10%: Pure intuition. Someone thinks it might work.
Effort
Effort captures the total cost of designing, developing, QA-ing, and running the test — expressed in person-weeks. A simple copy test that takes a copywriter half a day and a developer an hour might score a 0.5. A full checkout redesign requiring design, front-end development, back-end logic, and multi-device QA might score a 6.
The critical principle: Effort sits in the denominator. Higher-effort tests need proportionally higher Reach, Impact, and Confidence to justify their slot in the backlog.
Scoring in Practice: Three Real Examples
These are representative examples drawn from Converzion Catalyst client engagements, with commercially sensitive details adjusted.
Test A: Adding Trust Badges to Checkout
RICE Score: (85,000 × 3 × 0.8) / 0.5 = 408,000
High reach, moderate impact, strong evidence, and trivial effort. This test should be near the top of any backlog.
Test B: Complete PDP Redesign
RICE Score: (140,000 × 4 × 0.5) / 5 = 56,000
Despite massive Reach and high Impact, the moderate Confidence and heavy Effort drag this down. It's not a bad test — but it shouldn't leapfrog easier wins.
Test C: Homepage Hero Copy Variant
RICE Score: (200,000 × 1 × 0.3) / 0.3 = 200,000
Looks high on paper due to Reach and low Effort. But here's the trap: the Impact is 1 and the Confidence is 30%. Even if this test "wins," the actual revenue gain is likely negligible. RICE flags it as a candidate, but an experienced CRO practitioner should apply judgement here and deprioritise it in favour of tests with higher Impact and Confidence.
This is an important nuance. RICE is a prioritisation heuristic, not an autopilot. The scores create a rank order that you then apply strategic judgement to.
Building the Prioritised Backlog
Once every test idea has a RICE score, the backlog builds itself. But a backlog isn't a roadmap until it has structure. Here's the system Converzion Catalyst uses across client engagements.
Step 1: Score Everything
Every test idea — whether it came from analytics, user research, stakeholder input, or heuristic audit — goes through the RICE framework. No exceptions. The brainstorm that sounded brilliant on Tuesday gets the same scrutiny as the data-backed hypothesis from the analytics team. A spreadsheet with columns for each RICE dimension, the calculated score, the hypothesis statement, and the primary metric is all that's needed.
Step 2: Tier the Backlog
Sort by RICE score, then divide into three tiers:
- Tier 1 (Top 20%): Highest-score tests. These run first. Non-negotiable.
- Tier 2 (Middle 40%): Strong tests that queue behind Tier 1. Many of these will move to Tier 1 as learnings from earlier tests increase Confidence scores.
- Tier 3 (Bottom 40%): Low-priority ideas. Parked unless new data elevates them. Some will be retired entirely.
Step 3: Run in Structured Sprints
Testing works best in 2-week sprint cycles. Each sprint has a defined number of test slots based on available traffic and concurrent test capacity. For most mid-market eCommerce brands, that's 2–4 concurrent tests.
Every sprint starts with a planning session where the top-scoring tests from the backlog are assigned. Every sprint ends with a review where results are analysed, learnings documented, and the backlog rescored based on new data.
The most valuable output of a testing programme isn't the winning variant. It's the compounding knowledge base. Every test — win or lose — should generate an insight that makes the next test smarter. If your testing programme isn't building institutional knowledge, it's just generating random noise.
Statistical Significance: The Non-Negotiable Threshold
No discussion of CRO prioritisation is complete without addressing statistical rigour, because the most well-prioritised test in the world is useless if the results can't be trusted.
The 95% Confidence Threshold
Converzion Catalyst does not call a test result valid below 95% statistical significance. Some teams drop to 90% or even 85% to "move faster." This is a false economy. At 85% significance, there's a 15% chance the observed difference is random noise. Run 10 tests at that threshold, and statistically, 1–2 of your "winners" aren't winners at all. Implement them, and you've just degraded the experience for every user based on a coin flip.
Minimum Sample Size
Before launching any test, calculate the required sample size using the baseline conversion rate, the minimum detectable effect (MDE), and the desired significance level. For a page converting at 3% where a 10% relative improvement (3% → 3.3%) would be commercially meaningful, the required sample size is approximately 35,000 visitors per variation.
That means a simple A/B test (two variations) needs 70,000 visitors to the tested page to reach significance. At 500 daily visitors, that's 140 days — over four months. If your traffic can't support the test within a reasonable window (typically 2–6 weeks), one of two things needs to happen:
- Increase the MDE. Test bolder changes that would produce a larger lift, so fewer visitors are needed to detect it.
- Don't run the test. Seriously. Running a test you can't reach significance on is worse than not testing, because it produces false confidence in unreliable data.
Traffic Allocation Mistakes
The other common error is incorrect traffic allocation. Splitting traffic 50/50 between control and variant is standard for a simple A/B test. But when running multiple concurrent tests on the same user flow, traffic collision becomes a real problem. If Test A and Test B both affect the product page, and a user sees both variants simultaneously, neither result is clean.
The solution: mutually exclusive test audiences or sequential testing on shared flows. Converzion Catalyst uses traffic segmentation in tools like VWO and Convert to ensure test audiences don't overlap on the same funnel stage.
Compounding Learnings: The Real Competitive Advantage
Individual test wins are nice. Compounding learnings are transformative.
Here's what compounding looks like in practice. A Converzion Catalyst client in the fashion eCommerce space ran the following sequence over a 6-month testing programme:
- Test 1: Simplified the mobile checkout form from 11 fields to 6. Result: +18% checkout completion rate.
- Test 2: Added trust badges and secure payment indicators below the payment form (informed by Test 1 heatmap data showing hesitation at the payment step). Result: +9% checkout completion on top of the previous gain.
- Test 3: Introduced express checkout (Apple Pay, Shop Pay) as the default option for returning mobile users (informed by Test 2 data showing returning users still abandoning at payment entry). Result: +14% checkout completion for the returning user segment.
Each test informed the next. The cumulative effect wasn't 18% + 9% + 14% (additive). It was compounded: the checkout completion rate moved from 24% to 41% across six months. That's a 71% relative improvement — translating to approximately £1.4M in additional annual revenue for this particular client.
None of those tests would have been sequenced correctly without the RICE framework directing the order. And none of the insights from earlier tests would have been captured without a structured learning repository.
The First 30 Days: Your Implementation Plan
For teams ready to move from ad-hoc testing to RICE-prioritised CRO, here's a concrete 30-day implementation sequence.
Days 1–5: Audit and Inventory
- Compile every existing test idea, stakeholder request, and analytics observation into a single spreadsheet.
- Run a heuristic audit of the top 5 pages by traffic volume. (See: The 12-Point Heuristic Framework.)
- Pull baseline conversion rates and traffic volumes from GA4 for every key page and flow.
- Document existing test results and learnings (if any exist).
Days 6–10: Score and Prioritise
- Apply RICE scoring to every idea in the inventory. Involve the team — Confidence scoring works better as a group exercise.
- Tier the backlog into Tier 1 (run immediately), Tier 2 (queue), and Tier 3 (park).
- Calculate minimum sample sizes for every Tier 1 test. Flag any that can't reach significance within 4 weeks.
Days 11–15: Infrastructure Check
- Confirm your testing tool (VWO, Optimizely, Convert, Google Optimize successor) is correctly installed and tracking.
- Verify GA4 event tracking covers the primary and secondary metrics for your Tier 1 tests.
- Set up a testing dashboard with real-time significance monitoring.
- Establish traffic allocation rules to prevent test collision.
Days 16–25: Launch Sprint 1
- Launch your top 2–3 Tier 1 tests simultaneously (assuming no audience overlap).
- Monitor daily for data quality issues (tracking errors, sample ratio mismatch, flicker effects).
- Do not peek at results and make early calls. Let the test run to the pre-calculated sample size.
Days 26–30: Review and Iterate
- Analyse results at or above the 95% significance threshold.
- Document learnings — what worked, what didn't, and what new hypotheses emerged.
- Rescore the backlog. Some Tier 2 tests will now have higher Confidence based on Sprint 1 data.
- Plan Sprint 2.
The brands that win at CRO aren't the ones running the most tests. They're the ones running the right tests, in the right order, with the right sample sizes — and feeding every result back into a system that gets smarter with each cycle. RICE is the mechanism that makes that possible.
Testing without prioritisation is experimentation theatre. It looks productive. It generates charts and reports. But it doesn't compound, and it doesn't systematically grow revenue. RICE changes that — not because the formula is complex, but because it forces the discipline that separates programmes producing 3% annual improvement from those producing 30%+.
The maths isn't difficult. The discipline is. Start there.