Computational Marketing
FeaturesLong read

What a conversion rate optimisation agency fee actually buys: how a technical founder should scope CRO as instrumented experiments, not a retainer

A CRO fee buys test conditions, so audit your funnel first.

· 8 min read
Cover illustration for “What a conversion rate optimisation agency fee actually buys: how a technical founder should scope CRO as instrumented experiments, not a retainer”
Features · November 20, 2026 · 8 min read · 1,910 words

A conversion rate optimisation agency fee buys the conditions for experiments. For a technical founder, the decision starts with whether the signup-to-payment path can be measured and changes can be shipped. Comparing monthly fees comes later. The useful frame is instrumented experiments with clear outputs, rather than an ongoing relationship measured in hours.

Why a CRO retainer is priced around conditions, not tests

A retainer may be sold as tests per month, but the contract pays for the team’s time, the research cadence, and the testing infrastructure. A test needs a defensible hypothesis, a working implementation, and enough evidence to support a decision. If any of those is missing, the agency can stay busy without producing a trustworthy result. The structural mismatch is between the activity being sold and the evidence the founder needs.

Audits, fixed projects, monthly retainers, and performance-based arrangements reward different kinds of work, so the deliverable written into the contract is more useful than the pricing category. An audit can end with a diagnosis. A retainer funds the ability to keep researching, implementing, and learning as the funnel changes. Buying that ongoing capability before knowing what is broken can leave the first months devoted to defining the work.

The quoted fee also has a boundary. Third-party tool licenses, developer hours, and analytics cleanup often cost extra, even when experiments need all three. A founder should therefore budget materially above the stated retainer when estimating the full program cost. Otherwise, a proposal that seems affordable can rely on work that has no money or time from the internal team set aside for it.

Performance pricing seems to align payment with results, but attribution complicates the bargain. Conversion rates change for reasons outside an agency’s control, so deals based only on performance are rare and disputes are to be expected. This model can also reward flashy tests with results measurable this quarter over foundational work. A fair base with a modest performance component is the more workable version. A diagnosis is a project, while a capability is a retainer.

Why most CRO programs fail before the first test runs

Things often first go wrong when the agency forms the hypothesis. An agency inherits the client’s analytics and event tracking, including whatever gaps make the funnel hard to read. If that layer is broken or absent, the early weeks of the engagement go into paid cleanup. Without a reliable account of where users leave, a proposed experiment may focus on easy changes while leaving barriers to payment unaddressed. Testing skill cannot repair that missing premise.

The CRO value chain includes user research, hypothesis development, A/B test design, implementation, statistical analysis, results synthesis, roadmap management, and strategic reporting. A strategy-only agency handles the first three and last two components, leaving implementation to the client’s team. During a pitch, it is easy to miss who does which parts of the work. If the statement of work does not say who builds the changes, the proposal leaves a central condition of delivery unresolved.

Traffic constrains what the program can learn next. Below roughly a thousand visitors per variant per week, live A/B testing is statistically unreliable. That makes a steady testing cadence an unsuitable promise for some small sites, regardless of the agency’s skill. For low-traffic sites, implementation-first CRO offers a more honest model by shipping changes with the strongest evidence base and reading results directionally. A small-store proposal led by a testing-tool subscription has skipped this traffic conversation.

The revenue calculation places a further limit on the proposal. Divide monthly revenue by the current conversion rate, multiply by the target rate, then subtract current revenue and compare the difference with the fee. The target is an assumption. Still, the calculation shows whether the proposed improvement could justify the expense even on its own terms. If the economics do not clear that threshold, a cheaper test or a better presentation will not fix the underlying case.

What to instrument before any test runs

The practical response is to own and verify the instrumentation before bringing in an agency. Start with a stage-by-stage funnel covering meaningful actions from the first page view through revenue and CRM status. For a lead-led flow, the taxonomy includes landing page view, CTA click, form start, form submit, thank-you page, CRM lead created, SQL, opportunity, and revenue. Each stage needs its own conversion rate. A single headline rate hides where the loss occurs and makes the hypothesis harder to ground.

Next, check that analytics records user events at each stage of the key flows. The stage names alone are not instrumentation. Session recordings and heatmaps show more of what users do, while form-field abandonment analytics and scroll depth help show where users drop away before the agency forms a hypothesis. These views serve different purposes, so a recording should not substitute for a measured transition through the funnel.

For a signup-to-payment path, more signups are not the same output as more paid accounts. Track checkout completion and revenue per session with the headline conversion rate. For B2B, include lead-quality measures such as demo-to-SQL rate to distinguish successful leads among form fills. Guardrail metrics set the conditions that must not degrade even when the primary metric improves.

Before launch, calculate the minimum detectable effect, or MDE. This is the smallest improvement a test can reliably detect at the current traffic volume. A test that cannot detect a meaningful improvement wastes budget even if its design is otherwise sound. Making this calculation before work begins also exposes when the proposed experiment asks more of the traffic than it can support.

Finally, keep the measurement layer in accounts the company owns. If the data lives in the agency’s account, continuity ends with the engagement. Google Optimize sunset in September 2023. A founder should check whether the tier being purchased is expected to remain available in a year, rather than treating access to a testing tool as a permanent part of the stack.

Questions to ask a CRO agency before signing

How many tests has the agency documented, and can it show anonymized examples from the same industry? Data from past tests gives hypotheses more to build on than workshop exercises alone. The useful evidence is whether the agency can show which patterns hold in the relevant vertical. A case study by itself cannot answer that question.

How does the agency define statistical significance? What is its minimum sample size policy? An answer to trust gives a clear confidence threshold and explains how the agency calculates the required sample sizes before a test begins. Ask for a test log that includes losers as well. A win rate above half, or unwillingness to show failed tests, is a red flag because selected winners reveal less about methodology than the full record.

Who will be the day-to-day analyst, how much testing experience do they have, and how many other accounts do they run? The person delivering the pitch may not deliver the work. Ask who implements changes too, so the answer matches the staffing and capacity assumed by the proposal. These questions turn a general promise of expertise into an identifiable delivery arrangement. These questions also show whether the internal development team has been given work without having agreed to it.

Which metric will the agency optimize? A strong answer is revenue per visitor, with conversion rate and average order value looked at together. Walk away from an answer limited to conversion rate. Then ask who reviews AI output before it reaches the client and what the override process is. If AI writes the hypothesis, builds the test, and evaluates the result, the system checks itself unless a separate review is part of the workflow.

Whose account and license will hold the tools and data? The answer should be unambiguous, with ownership retained after the engagement ends. Ask what is not included in the quoted price, specifically tool licenses, developer hours, design, and analytics cleanup. Exclusions are part of scope. Funding for the proposed cadence and the client’s ability to continue the work after leaving depend on these exclusions.

The remaining questions belong in the contract itself. What defines the baseline, and is it a live control or a before-and-after comparison? A live control is cleaner because a before-and-after result is easier to dispute. What confidence level and minimum run time define a win, and what happens to a losing test? A change that does not beat the control should not ship. The terms should also state the minimum contract length, exit conditions, and attribution window for any performance-component fee.

Retainer vs metered primitives, compared on what each reports

The two models make different things visible to the buyer. A retainer purchases team time per month and commonly reports tests launched and tests “won,” with the risk that activity stands in for outcomes. Its typical minimum commitment is three to six months, extending up to a year for enterprise engagements. A founder weighing this model needs to know whether the report shows how the work paid for helps guide decisions about paid accounts.

Metered, per-test work pays for a hypothesis-to-significance read for each experiment. Its reporting centers on MDE, whether the required sample size was met, and guardrail metrics. The commitment is test-by-test, with the ability to pause. Setup time and qualitative research remain costs, however, and low traffic can still leave a test under-powered.

A retainer fits genuinely iterative optimization with enough traffic to sustain a steady cadence. Month three of a good program should not look like month one because earlier results should inform later work. For lower-traffic B2B SaaS products with bounded, identifiable funnel leaks, a fixed-fee audit followed by per-primitive execution is often better value. That does not make a statistical verdict available for every change. When there is too little traffic for testing, the report must make clear how directional evidence differs from a significant result.

PostHog Experiments places A/B testing next to analytics, session replay, and feature flags, without exporting data to a separate statistics layer. VWO and AB Tasty merged in January 2026, while Intellimize became part of Webflow. As vendors join together, there may be fewer dashboards but also more vendors going after larger customers. A platform subscription supplies the testing tool, whereas an agency supplies the program around it. The choice depends on whether the internal team can own that program layer.

AI-augmented work is another choice, as long as it is clear how the work is measured and reviewed. For teams that also need content infrastructure, a system can publish at a cadence, track whether AI answer engines name and cite the brand, and allow review before each output ships. That addresses discovery before the click, which neither an on-site CRO retainer nor a testing tool covers. Measuring mentions in ChatGPT, Claude, Gemini, and Perplexity extends the experiment-based approach to that surface. It is a distinct output that requires separate paid-account measurement.

Content that earns citations in AI answers can compound by creating retrievable, owned ground that persists across engine updates independently of any one prompt or hero page. That gives the founder another measurable surface, but it also sharpens the need to define what each purchase delivers. A visibility report answers a different question from an experiment read or a revenue result. Before signing, the useful question is therefore not simply how much work a monthly fee buys, but whether the proposed work can produce evidence the business is equipped to act on.

More in Features