Here's a scenario that plays out across nearly every eCommerce and SaaS brand running a testing programme: the team sits down on Monday, someone suggests changing the CTA colour on the product page, someone else wants to test a new hero banner, and the most senior person in the room lobbies for a pricing page redesign. Three tests get queued. None of them are connected to a hypothesis. None have been scored against potential impact. And by the time results come in six weeks later, nobody can articulate what was learned — or what to do next.

This is the testing treadmill, and most organisations are stuck on it. Research from Speero (formerly CXL Institute) shows that 60–70% of testing capacity at the average company is spent on low-impact or inconclusive experiments. That's not just wasted dev hours. It's wasted opportunity cost — the revenue that a properly prioritised test could have generated in the same window.

The fix isn't more tools or more traffic. It's a scoring framework that forces discipline into what gets tested and in what order. At Converzion Catalyst, that framework is RICE.

What RICE Actually Is (And Why It Works for CRO)

RICE was originally developed by Intercom's product team to prioritise feature requests. It stands for Reach, Impact, Confidence, and Effort. The formula is straightforward:

RICE Score = (Reach × Impact × Confidence) / Effort

The reason it transfers so well to CRO is that conversion testing shares the same fundamental challenge as product development: too many ideas, not enough capacity, and a dangerous temptation to go with gut feeling instead of structured evaluation.

Let's break each component down as it applies to conversion rate optimisation specifically.

Reach

Reach measures how many users will be exposed to the test variant within a defined time period — typically per month or per test cycle. This isn't the same as total site traffic. It's the traffic to the specific page or flow being tested.

A homepage test on a site doing 200,000 monthly sessions might have a Reach of 200,000. A test on the returns policy page might have a Reach of 4,500. The difference in potential revenue impact between those two is enormous, yet teams routinely treat all tests as equal.

How to score it: Pull the monthly unique pageviews or session count for the page or flow in question from GA4. Use the raw number. Don't normalise it.

Impact

Impact estimates the magnitude of the expected conversion lift per user who sees the variant. This is where hypothesis quality matters. A button colour change might shift behaviour marginally. Restructuring the entire checkout flow could double conversions.

Converzion Catalyst uses a 1–5 scale:

Confidence

Confidence is the honesty check. It forces the team to quantify how much supporting evidence exists for the hypothesis. A test idea backed by heatmap data, user recordings, survey feedback, and competitor analysis gets a higher Confidence score than one that came from a brainstorm.

Score as a percentage:

Effort

Effort captures the total cost of designing, developing, QA-ing, and running the test — expressed in person-weeks. A simple copy test that takes a copywriter half a day and a developer an hour might score a 0.5. A full checkout redesign requiring design, front-end development, back-end logic, and multi-device QA might score a 6.

The critical principle: Effort sits in the denominator. Higher-effort tests need proportionally higher Reach, Impact, and Confidence to justify their slot in the backlog.

Scoring in Practice: Three Real Examples

These are representative examples drawn from Converzion Catalyst client engagements, with commercially sensitive details adjusted.

Test A: Adding Trust Badges to Checkout

85K
Reach (monthly users reaching checkout)
3
Impact (medium — addresses trust gap)
80%
Confidence (heatmaps show hesitation at payment)
0.5
Effort (person-weeks)

RICE Score: (85,000 × 3 × 0.8) / 0.5 = 408,000

High reach, moderate impact, strong evidence, and trivial effort. This test should be near the top of any backlog.

Test B: Complete PDP Redesign

140K
Reach (monthly PDP views)
4
Impact (high — full layout restructure)
50%
Confidence (competitor analysis only)
5
Effort (person-weeks)

RICE Score: (140,000 × 4 × 0.5) / 5 = 56,000

Despite massive Reach and high Impact, the moderate Confidence and heavy Effort drag this down. It's not a bad test — but it shouldn't leapfrog easier wins.

Test C: Homepage Hero Copy Variant

200K
Reach (monthly homepage sessions)
1
Impact (minimal — headline tweak)
30%
Confidence (gut feeling)
0.3
Effort (person-weeks)

RICE Score: (200,000 × 1 × 0.3) / 0.3 = 200,000

Looks high on paper due to Reach and low Effort. But here's the trap: the Impact is 1 and the Confidence is 30%. Even if this test "wins," the actual revenue gain is likely negligible. RICE flags it as a candidate, but an experienced CRO practitioner should apply judgement here and deprioritise it in favour of tests with higher Impact and Confidence.

This is an important nuance. RICE is a prioritisation heuristic, not an autopilot. The scores create a rank order that you then apply strategic judgement to.

Building the Prioritised Backlog

Once every test idea has a RICE score, the backlog builds itself. But a backlog isn't a roadmap until it has structure. Here's the system Converzion Catalyst uses across client engagements.

Step 1: Score Everything

Every test idea — whether it came from analytics, user research, stakeholder input, or heuristic audit — goes through the RICE framework. No exceptions. The brainstorm that sounded brilliant on Tuesday gets the same scrutiny as the data-backed hypothesis from the analytics team. A spreadsheet with columns for each RICE dimension, the calculated score, the hypothesis statement, and the primary metric is all that's needed.

Step 2: Tier the Backlog

Sort by RICE score, then divide into three tiers:

Step 3: Run in Structured Sprints

Testing works best in 2-week sprint cycles. Each sprint has a defined number of test slots based on available traffic and concurrent test capacity. For most mid-market eCommerce brands, that's 2–4 concurrent tests.

Every sprint starts with a planning session where the top-scoring tests from the backlog are assigned. Every sprint ends with a review where results are analysed, learnings documented, and the backlog rescored based on new data.

Converzion Catalyst Principle

The most valuable output of a testing programme isn't the winning variant. It's the compounding knowledge base. Every test — win or lose — should generate an insight that makes the next test smarter. If your testing programme isn't building institutional knowledge, it's just generating random noise.

Statistical Significance: The Non-Negotiable Threshold

No discussion of CRO prioritisation is complete without addressing statistical rigour, because the most well-prioritised test in the world is useless if the results can't be trusted.

The 95% Confidence Threshold

Converzion Catalyst does not call a test result valid below 95% statistical significance. Some teams drop to 90% or even 85% to "move faster." This is a false economy. At 85% significance, there's a 15% chance the observed difference is random noise. Run 10 tests at that threshold, and statistically, 1–2 of your "winners" aren't winners at all. Implement them, and you've just degraded the experience for every user based on a coin flip.

Minimum Sample Size

Before launching any test, calculate the required sample size using the baseline conversion rate, the minimum detectable effect (MDE), and the desired significance level. For a page converting at 3% where a 10% relative improvement (3% → 3.3%) would be commercially meaningful, the required sample size is approximately 35,000 visitors per variation.

That means a simple A/B test (two variations) needs 70,000 visitors to the tested page to reach significance. At 500 daily visitors, that's 140 days — over four months. If your traffic can't support the test within a reasonable window (typically 2–6 weeks), one of two things needs to happen:

  1. Increase the MDE. Test bolder changes that would produce a larger lift, so fewer visitors are needed to detect it.
  2. Don't run the test. Seriously. Running a test you can't reach significance on is worse than not testing, because it produces false confidence in unreliable data.

Traffic Allocation Mistakes

The other common error is incorrect traffic allocation. Splitting traffic 50/50 between control and variant is standard for a simple A/B test. But when running multiple concurrent tests on the same user flow, traffic collision becomes a real problem. If Test A and Test B both affect the product page, and a user sees both variants simultaneously, neither result is clean.

The solution: mutually exclusive test audiences or sequential testing on shared flows. Converzion Catalyst uses traffic segmentation in tools like VWO and Convert to ensure test audiences don't overlap on the same funnel stage.

Compounding Learnings: The Real Competitive Advantage

Individual test wins are nice. Compounding learnings are transformative.

Here's what compounding looks like in practice. A Converzion Catalyst client in the fashion eCommerce space ran the following sequence over a 6-month testing programme:

  1. Test 1: Simplified the mobile checkout form from 11 fields to 6. Result: +18% checkout completion rate.
  2. Test 2: Added trust badges and secure payment indicators below the payment form (informed by Test 1 heatmap data showing hesitation at the payment step). Result: +9% checkout completion on top of the previous gain.
  3. Test 3: Introduced express checkout (Apple Pay, Shop Pay) as the default option for returning mobile users (informed by Test 2 data showing returning users still abandoning at payment entry). Result: +14% checkout completion for the returning user segment.

Each test informed the next. The cumulative effect wasn't 18% + 9% + 14% (additive). It was compounded: the checkout completion rate moved from 24% to 41% across six months. That's a 71% relative improvement — translating to approximately £1.4M in additional annual revenue for this particular client.

None of those tests would have been sequenced correctly without the RICE framework directing the order. And none of the insights from earlier tests would have been captured without a structured learning repository.

The First 30 Days: Your Implementation Plan

For teams ready to move from ad-hoc testing to RICE-prioritised CRO, here's a concrete 30-day implementation sequence.

Days 1–5: Audit and Inventory

Days 6–10: Score and Prioritise

Days 11–15: Infrastructure Check

Days 16–25: Launch Sprint 1

Days 26–30: Review and Iterate

Key Takeaway

The brands that win at CRO aren't the ones running the most tests. They're the ones running the right tests, in the right order, with the right sample sizes — and feeding every result back into a system that gets smarter with each cycle. RICE is the mechanism that makes that possible.

Testing without prioritisation is experimentation theatre. It looks productive. It generates charts and reports. But it doesn't compound, and it doesn't systematically grow revenue. RICE changes that — not because the formula is complex, but because it forces the discipline that separates programmes producing 3% annual improvement from those producing 30%+.

The maths isn't difficult. The discipline is. Start there.

Next Article → The 7 Checkout Friction Points Costing eCommerce Brands Up to 68% of Revenue