Log #026 | How startups should measure Marketing

Every startup wonders the same thing: "How do I know if my Marketing is working?" Seems like such a simple question, but the answer is not. If it was, then Martech wouldn’t be a multi billion industry.

Worse, the marketing measurement industry has made billions selling you the lie that you need fancy attribution models, complex dashboards, and a team of data scientists to understand if your Marketing works. Nope. I’ve now worked at one of the best marketing science teams in the world and then 4 startups in a row. Surprisingly, it’s startups that think marketing measurement needs to be more complicated than it needs to be and it happens for 3 reasons.

Illusions of:

When it comes to measurement, startups try to copy what big companies do. I understand the impulse. You want to be like Google, Meta, and Uber, but it's like a 10-year-old kid trying to bench press 300 pounds because they saw The Rock do it.

What’s even crazier is that startups try to copy what big companies do without understanding what they do. So, let me pull back the curtain a bit.

Yes, large companies have complex systems and data scientists churning out models and all that, but at the simplest level, it’s how much did we spend and what did we get from it. That’s what every MBR and QBR is about. Imagine being in a room with 20 senior leaders and going into the fact that you’re using a bayesian time series with something something covariate adjustments blah blah colinearity . Doesn’t happen. All of that magic is behind the scenes.

At these larger companies, complex decisions are still being made with blended metrics because we understand the nuances behind these blended metrics. At Uber, we knew that paid represented ~X% of our Blended CAC. We did that with incrementality testing and MMMs and other tools but we didn’t sit there and calculate a paid CAC and then our organic contribution…. We used a BLENDED CAC.

When you’re a startup and you’re in survival mode, every $ counts. I can only empathize with founders who have taken the risk to start something they believe in and invest their entire focus on this one mission. So, naturally, founders tend to be very focused on marketing efficiency because it’s the biggest variable expense they’ll have every month.

In addition, Marketing expenses are a necessity for growth unless you have crazy virality but those are just the exception. Startups must spend money to make money so when you have an expensive necessity, you tend to be on top of it.

The illusion of control comes into play when the tools you use to measure marketing efficiency appear to be more robust and accurate than they truly are.

Enter “In platform” measurement.

Both Meta and Google freely give away robust measurement tools that help you understand what’s happening to your spend. “Oh wow! How kind of you Google!”

Now, why would two trillion dollar companies give away anything for free? WHAT?! They have ulterior motives. The horror.

Yes, in platform attribution is designed to make you spend more. It’s literally a casino. No windows (black box models). Free spins (give us your budget and we’ll do the optimization). Free drinks.

And now that Apple has become more annoying about measurement, companies have introduced proprietary probabilistic models that help you overcome them. Don’t get me wrong… these tools can be useful but it’s important to understand the business goal behind them. Just like you A/B test changes, so are Google and Meta on their “In Platform Measurement” to see what resonates more with you.