Before You Scale Retail Media, Define the Counterfactual

A retail media dashboard can show strong attributed revenue while leaving the next budget decision unresolved. The question is what customers would have purchased without the additional advertising. That alternative outcome is the counterfactual, and it belongs in the plan before a campaign starts.

IAB and IAB Europe's November 2025 commerce media guidelines describe several measurement approaches, including experiments, model-based counterfactuals, econometric models and hybrid proxies. Their overview emphasizes credible comparisons, bias control and separating signal from noise. The brief below is Aeris's practical interpretation for teams deciding whether to expand a campaign.

Write the budget decision first

Start with a sentence the business can act on: we want to know whether increasing investment in this placement creates enough additional contribution to justify the cost. Specify the product group, audience and market.

This is more useful than a general ambition to prove that retail media works. A channel can produce value in one situation and have limited additional value in another. The decision should be narrow enough that the result changes an actual allocation.

Also distinguish a new campaign from an increase to an existing campaign. Measuring the effect of all advertising does not automatically tell you the return from the next unit of spend.

Choose the outcome before the method

Agree on the primary outcome and how it will be calculated. Gross sales, retained net sales, new customers and contribution answer different questions. A campaign can improve one while weakening another.

For a commerce business, follow the outcome far enough to account for cancellations and returns. Define what qualifies as a new customer and the historical lookback used to identify one. Keep the outcome definition consistent across the groups being compared.

Choose a small number of secondary measures to explain the result. They should help diagnose changes in basket size, product mix or repeat purchases, rather than provide many chances to discover a flattering number.

Make the comparison credible

Where feasible, randomized treatment and control groups can support a clearer causal comparison. The design still matters: decide whether assignment happens at customer, geographic or another suitable level, and assess whether exposure can spill between groups.

When randomization is unavailable, a modeled comparison may be useful. Write down its assumptions, the historical data it needs and the reasons the comparison could fail. Similar past performance does not guarantee that two groups would have evolved identically during the test.

Ask the analyst to estimate the sample and duration needed for a commercially meaningful effect. A study that cannot distinguish a useful improvement from ordinary noise may be poorly suited to the budget decision, even if the reporting looks precise.

Record what could contaminate the result

Promotions, stockouts, pricing changes and other campaigns can alter demand while the media test is running. Document these events and decide in advance how major disruptions will be handled.

Avoid changing the test window after seeing early results. Repeatedly checking the data and stopping when it looks favorable can make apparent success misleading. Use a planned evaluation point or a statistical approach designed for sequential decisions.

Keep implementation checks separate from outcome checks. Confirm that the intended groups received the intended treatment, tracking remained consistent and inventory was available. A failed implementation is a reason to qualify the result, not to manufacture a confident conclusion.

Report economics and uncertainty together

Show attributed performance and estimated incremental performance as separate views. Explain the comparison, the uncertainty around the estimate and any important limitations. A positive point estimate alone is not the full decision.

Translate additional retained sales into contribution using an agreed margin definition, then account for the incremental campaign cost. Higher sales do not necessarily justify scaling when the promoted products have different economics.

If the result remains inconclusive, say what evidence would resolve it. That may mean a longer planned study, a different design or a narrower commercial question.

Finish with a next action

A useful test brief ends before launch with the conditions for expanding, holding or reducing investment. Those conditions should reflect the business's economics and tolerance for uncertainty.

At the review, return to that original decision. The value of incrementality measurement is a better allocation of scarce budget, supported by evidence the team can explain and reproduce.

Source

IAB and IAB Europe: Guidelines for Incremental Measurement in Commerce Media, November 3, 2025. The operating brief is Aeris editorial analysis.