Conversion Lift Tests vs Holdout Testing
In short: Both hold back a control group to measure incremental conversions, but they differ in who manages the split and for how long. A conversion lift test is a discrete, time-boxed experiment you request through the platform's own tooling, using its identity graph to keep the control clean. Holdout testing is a standing, advertiser-managed suppression of a slice of one audience list, usually retargeting or CRM, left running indefinitely. Request a conversion lift test for a specific one-time budget decision, and keep a holdout running as a permanent check on your highest-risk audience.
By the AdFlint research team · Fact-checked against current Google and Meta platform behavior · Last reviewed July 2026
Conversion Lift Tests
A platform-run randomized experiment that holds a control group out of your campaign and reports the incremental conversions attributable to it.
Meta and Google both operate these internally, splitting your target audience into test and control and using their own identity graph to keep the control clean. You request one when you need incremental numbers rather than attributed ones, usually before a budget decision. Two limits to hold onto: the platform is grading its own work, and results only cover that platform, so cross-channel effects stay invisible.
Full definitionHoldout Testing
Permanently excludes a randomly selected share of your audience from seeing ads, so their behavior serves as an ongoing baseline.
A fixed percentage of users or accounts is suppressed from targeting for the duration of the test, and the gap between held-out and exposed groups gives you the incremental effect. Advertisers keep a standing holdout on retargeting and CRM audiences, where reported returns are most likely to be double-counted demand. Watch for leakage: if the held-out group can still be reached by other campaigns, the baseline is contaminated.
Full definitionSide by side.
The differences that actually change what happens in your account.
| Conversion Lift Tests | Holdout Testing | |
|---|---|---|
| Who manages it | The platform, through its lift-testing product. | The advertiser, through audience exclusion or list segmentation. |
| Duration | Time-boxed, ending when the test reaches significance or its scheduled close. | Indefinite - stays suppressed for as long as you keep the campaign running. |
| Eligibility requirements | Minimum conversion volume and platform approval to qualify for a test. | None beyond having an audience list you can split and suppress. |
| Typical scope | Any campaign type on that platform, prospecting included. | Almost always retargeting or CRM audiences, where inflated returns are most likely. |
| Setup effort | Requesting the test and waiting on platform scheduling and sample accumulation. | Building an exclusion list once and leaving it in place. |
| Result format | A single readout at test close, reported by the platform. | An ongoing comparison you check periodically rather than a single verdict. |
| Contamination risk | The platform's own control leaking through another campaign it does not see. | Leakage from your own other campaigns, organic reach, or email still touching the held-out group. |
What actually separates them.
A conversion lift test is requested and run by the platform on its own schedule; a holdout is something you build and maintain yourself, on your own timeline.
Conversion lift tests can cover any campaign type the platform sells, including prospecting; holdout testing is almost always scoped narrowly to an audience you already have, like retargeting or CRM.
The conversion lift test ends and delivers one verdict; the holdout never formally ends, so its value comes from watching the gap stay consistent over multiple cycles.
Qualifying for a conversion lift test requires meeting the platform's conversion volume threshold; a holdout has no such gate, since you control the split yourself.
Leakage in a conversion lift test is the platform's problem to manage inside its own tracking; leakage in a holdout is entirely on you to police across every other campaign, organic channel, and email touch that could reach the excluded group.
Which one should you use?
Use Conversion Lift Tests when
- You want to test prospecting or a full campaign type, not just an existing retargeting or CRM audience.
- You have enough conversion volume to qualify for the platform's test product.
- You need a single, scheduled verdict for a specific decision rather than an ongoing signal.
- You are comfortable relying on the platform's own identity graph to manage the control group.
Use Holdout Testing when
- You want a permanent, low-effort check on whether your retargeting or CRM spend is truly incremental.
- You would rather manage the suppression yourself than wait on a platform's test scheduling and eligibility.
- You want a continuous signal you can revisit any time, not a one-time report.
- You are prepared to actively monitor for leakage across your own other campaigns and channels.
Common questions.
Do I need to request a holdout test from the platform like a conversion lift test?
No. A holdout is something you build yourself by excluding a slice of an audience list from targeting - it does not require platform approval or a minimum conversion threshold the way a conversion lift test does. That is also why it is easier to leave running indefinitely.
Can a holdout test replace a conversion lift test?
For retargeting and CRM specifically, a well-run holdout answers a very similar question. For prospecting or full-funnel campaigns, a holdout does not really apply the same way, since there is no pre-existing audience list to split, so a platform conversion lift test is the better fit.
Why did my conversion lift test come back inconclusive?
Most often the account did not have enough conversion volume in the test window to detect a real gap between exposed and control. Requesting a longer window or waiting until volume grows is usually the fix, rather than treating an inconclusive result as proof of no effect.
Or stop choosing between them.
AdFlint picks the setting, writes the ads, and keeps optimizing inside the Google and Meta accounts you already own.
Related comparisons
- A/B Testing vs Incrementality Testing
- A/B Testing vs Conversion Lift Tests
- A/B Testing vs Brand Lift Tests
- A/B Testing vs Geo Testing
- A/B Testing vs Holdout Testing
- A/B Testing vs Marketing Mix Modeling
- A/B Testing vs Multi-Touch Attribution
- Conversion Lift Tests vs Incrementality Testing