Zero conversion ad campaigns

When testing out new advertising channels, you’re likely to run at least one campaign that produces exactly zero conversions. Along the way, you might wonder: how quickly can I call it quits on a particular campaign before moving on to try another?

Rule of thumb

If your campaign has spent 3X your customer margin and has produced zero conversions, then you can be >95% confident that the campaign is unprofitable. Here, customer margin is defined as a customer’s lifetime revenue minus all non-advertising variable costs associated with servicing that customer.

It turns out that there is a direct relationship between spend on a campaign with zero conversions and confidence that the campaign is unprofitable:

• 95% confidence in unprofitability ⟷ 3X margin with 0 conversions
• 80% confidence in unprofitability ⟷ 1.6X margin with 0 conversions
• 50% confidence in unprofitability ⟷ 0.7X margin with 0 conversion

Mathematical backing

Suppose that we have built a website that sells widgets. A few variables that’ll be helpful:

• $W$: lifetime revenue from a purchasing customer, less all non-advertising variable costs
• $P$: probability that each visitor results in a new customer who buys a widget
• $V$: the volume of visitors that we receive for the above-defined budget of W

Breaking even

We’ll assume that all visitors in a campaign are equally likely to buy a widget. With this assumption, our purchase count will follow the binomial distribution with mean $VP$, as will our revenue with mean $WVP$.

Our campaign will be profitable if we earn more than $W$ dollars in customer margin by spending $W$ dollars on our campaign: $WVP > W$. By solving for $P$, we can see that the campaign will break even if $P = \frac{1}{V}$ and will be profitable if $P > \frac{1}{V}$. In layman’s terms, this basically means that if we spend $W$ on a campaign, at least one of our visitors needs to convert.

Confidence after N visitors

How large does $N$ have to be for us to falsely conclude that the campaign is unprofitable less than 5% of the time?

Because purchase count is distributed binomially, we’d expect to see zero purchases with probability $(1-P)^N$. Therefore, we will look for an $N$ such that $(1-P)^{N} < 0.05$. Before doing so, however, we’ll set $P$ such that the campaign is break even (i.e., $P = \frac{1}{V}$); we can do this because $(1 - P)^N \leq (1 - \frac{1}{V})^N$ for any profitable campaign, and because we’ll soon choose an $N$ such that $(1 - P)^N \leq (1 - \frac{1}{V})^N < 0.05$.

We’ll plan to purchase visitors as a multiple of our budget, meaning we’d like to set $N = KV$. It turns out that for all $K > 0$, $f(V) = (1 - \frac{1}{V})^{KV}$ is maximized with respect to $V$ as $V$ approaches infinity. To gain intuition for why that’s the case, check out a graph of $f(V)$ where $K = 1$.

By taking $\lim_{V \to \infty} f(V)$, we can then solve for $K$ such that the inequality is true for all values of $V$. After taking the limit, we see that $\frac{1}{e}^{K} < 0.05$. Solving for $K$, we end up with ln(20) = 2.9957, which is how we got to our 3X rule of thumb. Generalizing this technique, we see that to achieve a given $Z$ percent confidence that a campaign is in fact unprofitable after zero conversions, we need to spend $ln(\frac{1}{Z})$ multiplied by our customer’s lifetime margin.

Conclusions

This approach can give you a sense of how large a budget one should allocate to a test of a new ad platform, a new product, or a new landing page. Further, this approach works independent of whether you are buying mobile app installs, clicks, or impressions.

If your customers are typically worth $100 when they convert, then this rule of thumb tells you that the minimal rigorous test you can run on an advertising platform is in the ballpark of $200-$300. If you’d like to have the opportunity to run multiple campaigns (say, a dozen), you should expect to spend up to $3,600 (=$300*12) in exploratory budget.