November 12, 2026

Incrementality Testing: Finding Your Real Customer Acquisition Cost

Share on :
Table of Contents

Every marketing team has a number it trusts more than any other. For most, that number is customer acquisition cost. It sits at the top of the board deck and anchors every budget review. When CAC is low, a channel gets more money. When it's high, the channel gets cut. Incrementality testing asks a harder question that most teams never put to that number: did the marketing actually cause the sale?

The answer is often less flattering than the dashboard. According to Uri Einav, the problem usually starts with how the credit gets handed out. Most companies measure on last click or last view. The ad networks reporting those conversions are the same ones being paid for them, so a channel can look like a hero while it collects credit for customers who were already on their way.

In this episode of Performance Delivered, host Steffen Horst is joined by Uri Einav, CEO and founder of Markup, which offers fractional growth leadership for B2C companies and startups. Uri spent a decade as a CMO and was part of three exits in gaming and ad tech. He has led growth teams of more than 40 people and managed budgets of up to $5 million a month across more than 150 apps and D2C brands, spanning performance channels, out-of-home and events. Where the clicks ran out, incrementality was how he proved what worked. Steffen and Uri cover what incrementality really measures, how it changes the math behind your unit economics, and when it isn't worth doing.

Incrementality asks one question that attribution can't

Uri keeps the definition simple. Incrementality means asking yourself, "would this conversion happen if I hadn't spent the marketing dollars?" You're checking whether your spend produced any lift you wouldn't have gotten anyway.

Attribution answers a different question: which touchpoint gets the credit? Uri sorts the main approaches like this. Last click and last view are what roughly 95% of businesses run on. Multi-touch attribution scores several touchpoints along the funnel, from brand initiatives and influencers to search. Marketing mix modeling (MMM) is statistical. It builds a baseline from historical performance, predicts what "normal" should look like, and measures how your marketing efforts move you away from it.

Incrementality sits on the other side of all of these. It doesn't divide up credit. It measures lift in conversions or revenue.

The case for incrementality testing is that it's unbiased

Uri doesn't oversell the method. He doesn't claim incrementality beats every other model, and he thinks it makes the most sense as part of a mix. "The best thing I can say about incrementality is that it's unbiased," he says.

The purest version is a holdout: stop the marketing and watch what happens. A big drop tells you something was working, and then you can work out the math. A step down from that is a classic A/B setup, where a test group sees your ad and a control group sees nothing, or sees a deliberately irrelevant placeholder ad. Uri once used a static red-cross ad for this, "something that does not resonate with the product at all." You can also run a geo holdout by pausing marketing in one of two similar countries and comparing the results.

Because these tests don't rely on platform reporting, the results hold up. "No one ad network can tell you this is wrong," Uri says.

Channels with no clicks are where attribution breaks down

Attribution struggles when a channel produces no clicks, which is the case for CTV, billboards and much of influencer marketing. Incrementality has no such problem. "The good thing about incrementality is that it doesn't need clicks," Uri says. Instead of tracking clicks, views or impressions, it measures what changed after a specific action at a specific point in time.

Uri recently tested this on a subscription app. The team was running an Apple Search Ads brand campaign measured on both last click and last view. That meant someone who saw the ad without clicking, then found the app organically within the attribution window, still counted as a paid conversion. That makes the campaign a clear candidate for a holdout.

When they paused it, there was no incremental lift. Uri recommended switching the brand campaign to last click only. On last view, "you're probably overpaying for people that would download the app anyway."

Your real CAC is probably higher than your dashboard says

This part changes the numbers in the board deck. Last-click attribution can count customers you would have gotten anyway. Remove them, and the same spend is buying fewer customers.

"When you do a real incrementality test, the CAC may go higher and show you an actual source of truth," Uri says. That pulls your LTV to CAC ratio down, often below what the team is used to seeing. It also changes the criteria for scaling, holding or killing a channel. If a channel shows a $100 CAC on last click and incrementality confirms those conversions were real, nothing changes. If it doesn't, your true CAC is higher. But at least then "you have an actual LTV to CAC ratio that you can trust."

A lower ratio you can trust is worth more than a higher one you can't, because every budget decision is built on top of it.

Brand search and retargeting are the usual suspects

Every marketer has a channel they suspect is taking too much credit. Uri doesn't take long to name his. "The highest-intent channels are the ones that are my usual suspects," he says. That starts with search, and above all brand search, because "people looking for my brand are probably potential customers that would have come anyways."

Retargeting is next on his list. Uri often sees companies "preferring to do retargeting than to actually build a lifecycle, CRM, retention system," so they pay for conversions an owned channel could have driven.

Fraud is the other risk. Uri tells the story of the 1902 rat bounty in French colonial Hanoi. Officials paid for every rat tail handed in, and people soon started breeding rats to collect the reward. Paying fraudulent traffic sources per conversion creates the same incentive: they deliver whatever earns the payout. Uri says an incrementality test is "a really, really good way" to catch it. He calls himself "a skeptic by nature," and that is the habit he wants marketers to build. Don't just believe what you see on "self-attributing networks that are telling you that life is great."

Geo tests are the safest way to prove a new channel

Incrementality isn't only for auditing existing spend. It's also a lower-risk way to add new channels. Steffen describes the approach: before rolling out CTV nationwide, test it in a single market like Los Angeles while everything else stays the same. Expand only if it moves your KPIs.

Uri ran the same playbook for a large D2C brand in Europe. The brand ran billboards in one city ahead of a conference. The team built an MMM benchmark from that city's historical performance, measured the change while the billboards were up, then checked whether performance returned to baseline after the campaign ended. "You have to really eliminate all the variables," he says.

This matters most for channels that last touch can't see. "It is pretty much well known that 70% of the users that come from influencers would show up as organic," Uri says. They don't click the link in the bio. They go straight to the store or the website.

When incrementality testing is a waste of money

Uri is upfront about the costs. "Incrementality isn't for everyone." He also admits it isn't pleasant to run, because nobody wants to pause budget on a channel that looks great. "It's not the most fun in the world, but it is something that tells you the truth."

It doesn't pay off in three situations:

  • No baseline yet. If a product is new and has no brand or organic demand, MMM has no history to model, and an incrementality test will likely come back one-to-one.
  • Volumes too low to read. "If we're not talking in thousands of conversions, then it's very, very hard to measure."
  • Paid is the only thing fueling growth. If you're on one or two trusted channels and still proving product-market fit, stick with last-touch attribution.

His trigger for starting is a question: is this channel really helping me? Once you're running affiliates, influencers, out-of-home or other initiatives that drive organic traffic, it's time for incrementality testing, MMM, or both.

The right answer isn't always the one that keeps your job

Steffen closes by asking Uri about his biggest measurement mistake. Uri says he "can name dozens," but the story he tells is about a result that was right and still cost him the engagement.

A gaming client was spending heavily on a major premium channel while also pulling in strong organic installs through app store optimization. Uri's geo holdout showed the paid channel was mostly buying users the company was already getting for free, and paid wasn't ROAS-positive at day 90. He gave the CEO two choices: pause paid and double down on organic, or "keep going blind" and keep spending.

"He looked at the results, he made the right decision, and I was let go," Uri says. The company needed to focus on organic and rebuild the product, and that left no role for a growth lead. An honest test can be the most valuable thing a marketer delivers, even when it costs them the job.

Episode Transcript

‍

Steffen Horst (00:11)
Welcome back to Performance Delivered, Insider Secrets to Marketing Success, the podcast where we explore what's actually driving growth and performance in today's complex marketing landscape. I'm your host, Steffen Horst, and today we're taking a hard look at one of the most trusted numbers in marketing: customer acquisition cost. CAC shows up in every board deck and budget review. But when last-click attribution decides who gets the credit, some channels look like heroes and others look like dead weight, even when the opposite is true. And if your CAC is off, so is your LTV to CAC ratio, and every budget decision built on it. That's where incrementality comes in. Not what got the credit, but what actually caused the sale. Joining me today is Uri Einav. Uri is the CEO and founder of Markup, which offers fractional growth leadership for B2C companies and startups. He spent a decade as a CMO, was part of three exits in gaming and ad tech, and has led growth teams of more than 40 people. Across more than 150 apps and D2C brands, he's managed budgets of up to five million a month, spanning performance channels, out-of-home and events. And where the clicks run out, incrementality was how he proved what worked. Uri, welcome to Performance Delivered. It's great to have you on.

Uri Einav (01:37)
Thanks for having me. Super excited to be here.

Steffen Horst (01:39)
Let's start with a simple question. Most marketers have heard the word incrementality, but it means different things depending on who you ask. In plain terms, what is incrementality, and why isn't the last click enough on its own?

Uri Einav (01:55)
Okay, that's a good one. Basically, incrementality, to make it as simple as possible, is asking yourself: would this conversion happen if I hadn't spent the marketing dollars? That's basically the idea. We're checking if our marketing spend has given us any incremental lift, any extra that we wouldn't have gotten anyway. And in today's world, a lot of businesses, in every aspect of the business, B2C, B2B, whatever, measure success by looking at customer acquisition cost, which usually relies on last view or last click. And in many, many cases, the ad networks are biased into giving us that result of a conversion, while in some cases we would have gotten that conversion anyway.

Steffen Horst (03:00)
There's incrementality, and then there's a lot of talk about attribution. Can you pick both apart?

Uri Einav (03:07)
I get asked that a lot. Let's try to break it down into types of measurement. This is really high level; we're looking from a hawk's eye. There's last click or last touch, which is last click and last view, which is what, let's say, 95% of businesses are working on. There's the opposite side, which is incrementality, which is looking at your incremental lift in conversions or revenue. There's MMM, which is marketing mix modeling, which is basically a statistical way of looking at historic performance. You create some kind of benchmark or baseline based on historic performance, you model and predict what your normal should be, and then you see how that changed over time or with any efforts that you made in the marketing. And another way of measuring, which is also very popular, is multi-touch attribution. First you have first-touch attribution, which is the first click in the funnel, but also multi-touch, where you can measure, I don't know, like 10 points of engagement, from brand initiatives to influencers to search to last click, and then you score each one of these points differently.

Steffen Horst (04:34)
Where does incrementality potentially have an advantage over all the other models that you just outlined?

Uri Einav (04:42)
I wouldn't say that it has an amazing advantage over anything. I think it makes a lot of sense to put it in the mix and work with it. The best thing I can say about incrementality is that it's unbiased. The ideal incrementality test is just a holdout: basically stopping your marketing altogether and then seeing what happens. If there was a big drop, then it means that probably something in the marketing was working, and then you really understand that math. You could also do an incrementality test that's very basic: A/B testing, with a control group seeing your ad and another group, a test group, not seeing an ad at all. Or there's that old-fashioned way. I had an incrementality test where the test group had a red cross ad, a static ad, so really something that does not resonate with the product at all, or not seeing an ad at all, and then seeing what happens to that cohort, the control group with the marketing versus the other one.

And I think incrementality, first of all, is not for everyone. A product that just started, that doesn't have any baseline or any brand or any organic, would probably not work with incrementality; it would not show a lot. But let's say you have multi-channel or omnichannel, and you have a lot of initiatives in the marketing, including, you name it, programmatic, CTV, out-of-home, billboards, SEO, GEO, brand awareness, blah blah blah. If you have all that, then for every new initiative that you do, if you perform incrementality over it, besides your last click, which is a must, you'll probably get an unbiased picture of how the marketing was doing, what your new initiative was doing, what pausing the old one was doing. You can also do a geo holdout: take two countries that are similar, pause the marketing in one, and see how that goes. So it's a big way of testing and seeing what your marketing is doing, but in a very unbiased, very scientific way, and no one ad network can tell you this is wrong, because these are super unbiased.

Steffen Horst (07:37)
When people use attribution to identify what channels are doing, that model usually struggles when there is no click involved in the media buy. You've run budgets of five to ten million a month, and not all of those channels might produce clicks. How do you measure channels through incrementality when there are just no clicks to attribute?

Uri Einav (08:07)
The good thing about incrementality is that it doesn't need clicks. It's not measuring clicks at all. It's not measuring views or impressions or anything. It just measures what happened based on the points in time when you made changes. So an incrementality test could be whether a seasonal change gave incremental lift. No clicks were involved in any way. Incrementality doesn't even look at the clicks; it just looks at the initiatives that were done at that point in time and sees what happened during that time, during that cohort period that you are measuring: if it went up, went down, stabilized, or nothing happened. I can give you an example to see the difference.

Steffen Horst (08:59)
Please.

Uri Einav (09:00)
One of the worlds that I'm very active in, where I do a lot of consulting and a lot of fractional CMO work, is mobile apps. Recently, with my company Markup, we worked with a subscription app. We have a consultancy, and we were working as a fractional CMO. It's more of a one-man show of me entering inside the team and helping them out. We were looking at Apple Search Ads, which shows very high intent. This is search, same as Google search: people are searching for a similar app to yours, your competitor, your own brand, or stuff like that. And on user acquisition, they were doing a brand campaign that was measuring both last click and last view. That means that someone looked your brand up in the last, let's say, 24 hours, they just saw the ad, and then they went organically and downloaded it.

So that's an initiative that is a clear-cut case for an incrementality test. If I do a holdout, if I stop that marketing, do I see an incremental drop? Basically, is the marketing giving an incremental lift? Because this is last view, people are watching my ad, not clicking on it, and going organically through my own app store placement. So that would show real incremental lift if you do the holdout. We tried that, and it did show that there was no incremental lift with the brand campaign on last view. So my recommendation to the company was to start doing last click on the brand campaign. Because if you're looking at last view, you're probably overpaying for people that would download the app anyway.

Steffen Horst (11:11)
Okay. Now let's get into the math. CAC is the number everyone watches, and LTV to CAC is the ratio everyone uses to decide whether to scale. When you start measuring incrementality instead of attribution, how does your CAC change, and what does that do to your LTV to CAC ratio?

Uri Einav (11:49)
Last click basically could count customers you'd have gotten anyway, in some cases. So when you do a real incrementality test, the CAC may go higher and show you an actual source of truth. So the LTV to CAC would probably be lower than you're used to, but it shows you the actuals. And that changes the scale, hold, kill criteria for your marketing. When you look at last click or last view, your CAC is, let's say, a hundred. But then if you start doing incrementality, you see what actually happened: whether those conversions were incremental or not. If they were incremental, you stay where you are. But if it's higher, then your CAC goes higher, but then you have an actual LTV to CAC ratio that you can trust.

Steffen Horst (13:00)
Makes sense. Now, the practical side. There are a lot of ways to test incrementality: a geo holdout, which you talked about earlier, a platform lift test, simply pausing a channel, which you mentioned, and media mix modeling. They all come with different costs and different levels of rigor. Walk us through the options, and how would you pick one for a team working with a limited budget?

Uri Einav (13:24)
That's an awesome question, because first of all, as I said, incrementality isn't for everyone. If we're just getting started, if we have zero data, then marketing mix modeling wouldn't help you, because it needs a benchmark; it needs historic performance. And incrementality, when you're running on one or two channels in the beginning and you don't have any organic, is probably not going to show you any new results. It's probably going to be one to one. And I wouldn't waste that dollar, because incrementality, to be perfectly honest, is a bit of a pain in the ass for a marketing team. Having to pause your entire budget on something that looks to be amazing isn't fun. It's not the most fun in the world, but it is something that tells you the truth. It's a source of truth.

So what I would suggest is, first of all, if you're just getting started, if you're just on the go-to-market and trying to prove PMF, product-market fit, then the best approach would be to start with your last-touch attribution and get to a point where it looks amazing. And then when you start having a lot of organic or a lot of multi-channel marketing spend, when you start working with affiliates, when you start working with influencers, when you start initiatives that are not one-to-one and have users that would come organically, or when you get to a point where you ask yourself, is that initiative helping me a hundred percent? Am I sure that this is what's bringing the best users, and I'm not just getting free users? Then I would start incrementality testing or MMM or both. It's a matter of resources in the end.

Steffen Horst (15:34)
Makes sense. And you obviously need to have a multitude of channels in order to do some testing. If you only run one channel, it doesn't really make much sense to apply this. But I would assume, if you want to roll out a new channel and you want to see what the impact is, as you said earlier, you could just build it out in one part of a city or of a country, whatever you want to use from a geo perspective, and then just see what the impact is there. If there's a positive impact on your KPIs, then you probably would roll it out. And that's probably also a great way of expanding the channel mix and helping a company grow. If we take the US, for example: if you want to test how CTV impacts your campaign, whether it gets through to the bottom line and you see greater revenue generated, you probably don't want to start spending across the whole US. You might just say, "Hey, you know what, we'll take Los Angeles, for example, and test it there," if everything else is the same. And if that's positive, then we can start rolling it out.

Uri Einav (16:44)
And then you can really use it as incrementality and see if my CTV campaign was incremental. Or use MMM and see, "That was my benchmark in LA, and now this is..."

Steffen Horst (16:50)
Yep, exactly.

Uri Einav (16:53)
We did the same for a billboard campaign for one of the big brands I used to work with, a big D2C brand in Europe. We had a billboard campaign in one of the cities before a conference, and we said, "Okay, let's just do marketing mix modeling." We did a benchmark of what that city was doing over enough time, looked at historic performance, and then saw what had changed in that specific city during the time the billboards were there. And then again, to check ourselves, once we paused the billboard campaign, we looked to see if the benchmark had changed in any way, if it went back to normal.

So you have to really eliminate all the variables and test it outright to see that it does have some kind of lift. Because if you just measure last touch for such a campaign, an out-of-home campaign, for example, or even influencer campaigns in general, it is pretty much well known that 70% of the users that come from influencers would show up as organic. They wouldn't click your bio; they'd just go to the store or to the website and convert from there. So when you have multiple channels, this is the best way to start seeing the source of truth, and not just believe what you're seeing on those self-attributing networks that are telling you that life is great.

Steffen Horst (18:33)
Indeed. Every marketer has a channel they suspect is taking too much credit. In your experience, which channels most often claim sales that would not have happened anyway?

Uri Einav (18:46)
That's an easy one. I think the highest-intent channels are the ones that are my usual suspects. That would be, first of all, search, where the user has very high intent, and even higher in that ranking is brand search. People looking for my brand are probably potential customers that would have come anyways. Retargeting as well. In many, many cases, retargeting is money that I shouldn't have spent. In many cases I see companies that are preferring to do retargeting than to actually build a lifecycle, CRM, retention system. So that's also a usual suspect. And also when a channel is biased toward getting conversions... I'm a skeptic by nature. I don't usually believe what I see in the news, and that goes for business as well.

And you have a lot of fraud. You probably know the story about the Hanoi rat massacre. Have you heard about it?

Steffen Horst (20:05)
No.

Uri Einav (20:07)
In 1902 in Hanoi, Vietnam, which was a French colony, the French started building a new sewage system. For rats, that meant a party, so there were tons of rats in the streets, and they feared a plague. There was a plague, and again, no disrespect to anyone who died in those years. What the French did is offer a bounty: one cent for each rat tail, because they didn't want to get the bodies. So they said, "Just give me the tails and I'll give you a bounty of one cent." And what happened from there is that the French started seeing rats without tails walking the streets. Basically, people started breeding rats. They had rat farms; they just took rats, and the breeding just didn't stop.

So that's it in a nutshell. It really reminds me of fraudulent traffic that you sometimes buy, where you pay them a CPA for a conversion, and they just need to give you a conversion, so they do what it takes to get that incentive. I think that's something to look out for when you start working and you don't measure fraud. Sometimes an incrementality test is also a really, really good way to look at that.

Steffen Horst (21:42)
Now here's a more contrarian question. Incrementality testing takes time, money and patience. When is it actually a waste of money? You mentioned earlier that you need to have a certain amount of data in place for it to make sense. But when is it a waste of money?

Uri Einav (22:06)
I think when spend is too low to actually read a result, to actually understand if it was incremental or not. If you're looking at something that isn't statistically significant, for example, if your answer would be, "I got those ten, a hundred," I don't know. If we're not talking in thousands of conversions, then it's very, very hard to measure. If we're talking in units, then it's pretty much impossible to know if that helped or not. And in cases where paid marketing is a must, especially in the beginning, or with lower budgets where marketing spend is the only thing that fuels that ecosystem, then it's too early to check incrementality.

When you're running on one or two channels, premium channels, top channels that you trust, you could always do incrementality hands-on, without really measuring it: just try to look, or do MMM and try to work with a benchmark. I've done that, and you know, even asked Claude or ChatGPT if it looks incremental or not. But when you're looking into serious incrementality testing or MMM, the kind that goes along with last click or multi-touch, I would start when you're asking yourself: Is this channel helping me or not? Am I spending too much? Did opening it change anything? If you're still just getting started with a new product, it's probably not best to start.

Steffen Horst (23:55)
The last question I have for you: after running so many budgets across this many apps and brands, you've surely gotten one incrementality test wrong. What's the biggest mistake you've made measuring a channel, and what did it cost you?

Uri Einav (24:14)
Just one? I can name dozens of mistakes I've made. But one that was dead wrong is something that cost me a client. It cost me my job, so that was a hard paycheck to let go of. I had a client, and we were spending a lot on one of the bigger premium channels. The company had a huge organic following. We're talking about a gaming company, and they were really, really good with organic, with ASO. What I did is a geo holdout, and I proved that the premium channel was getting us users that we already had for free, that we shouldn't have paid for.

And I came to the CEO and told him, "You have two choices right now. I'm being super unbiased here. One option is to pause everything and double down on your organic, because it's bringing a lot more than your paid right now, and your paid isn't day-90 ROAS positive at the moment. Your other option is to keep your eyes shut, to keep going blind, keep spending, and tell yourself that you're bringing in a lot of conversions that way." He looked at the results, he made the right decision, and I was let go, because I didn't have more room to help them. They needed to cut everything down, specialize on the organic side, and then build the product again.

Steffen Horst (26:04)
Uri, thank you so much for joining me on Performance Delivered and sharing your perspective. It's clear that the number on the dashboard isn't always the truth. Real growth comes from knowing what actually causes a sale, building your unit economics on that, and putting your budget where it makes a real difference. For listeners who want to learn more about you and your work and connect with you, where's the best place to reach you?

Uri Einav (26:31)
I'm a very responsive guy. You can reach out on LinkedIn, talk to me on LinkedIn, or go to our website, markupcmo.com. I love true, real conversations. So whoever wants to talk, whoever has any concerns about their growth and wants to take it to the next level, I'm here for them.

Steffen Horst (26:54)
Perfect. As always, we'll leave that information in the show notes. Thanks, everyone, for tuning in. If you enjoyed this episode of Performance Delivered, please subscribe and leave us a review on iTunes or your favorite podcast platform. To learn more about Symphonic Digital, visit us at symphonicdigital.com. See you next time.