👋 Hey, it’s Sundar! Welcome to experiMENTAL: a weekly newsletter on B2C Marketing & data science how-to guides, frameworks, and stories from 15 years including early Uber.

This week is sponsored by Superhuman AI

Start learning AI in 2025

Keeping up with AI is hard – we get it!

That’s why over 1M professionals read Superhuman AI to stay ahead.

• Get daily AI news, tools, and tutorials

• Learn new AI skills you can use at work in 3 mins a day

• Become 10X more productive

Sign up and start mastering AI

Why startups complexify measurement

Every startup wonders the same thing: "How do I know if my Marketing is working?" Seems like such a simple question, but the answer is not. If it was, then Martech wouldn’t be a multi billion industry.

Worse, the marketing measurement industry has made billions selling you the lie that you need fancy attribution models, complex dashboards, and a team of data scientists to understand if your Marketing works. Nope. I’ve now worked at one of the best marketing science teams in the world and then 4 startups in a row. Surprisingly, it’s startups that think marketing measurement needs to be more complicated than it needs to be and it happens for 3 reasons.

Illusions of:

1. Complexity

2. Certainty

3. Control

The illusion of complexity

When it comes to measurement, startups try to copy what big companies do. I understand the impulse. You want to be like Google, Meta, and Uber, but it's like a 10-year-old kid trying to bench press 300 pounds because they saw The Rock do it.

What’s even crazier is that startups try to copy what big companies do without understanding what they do. So, let me pull back the curtain a bit.

Yes, large companies have complex systems and data scientists churning out models and all that, but at the simplest level, it’s how much did we spend and what did we get from it. That’s what every MBR and QBR is about. Imagine being in a room with 20 senior leaders and going into the fact that you’re using a bayesian time series with something something covariate adjustments blah blah colinearity . Doesn’t happen. All of that magic is behind the scenes.

At these larger companies, complex decisions are still being made with blended metrics because we understand the nuances behind these blended metrics. At Uber, we knew that paid represented ~X% of our Blended CAC. We did that with incrementality testing and MMMs and other tools but we didn’t sit there and calculate a paid CAC and then our organic contribution…. We used a BLENDED CAC.

The illusion of control

When you’re a startup and you’re in survival mode, every $ counts. I can only empathize with founders who have taken the risk to start something they believe in and invest their entire focus on this one mission. So, naturally, founders tend to be very focused on marketing efficiency because it’s the biggest variable expense they’ll have every month.

In addition, Marketing expenses are a necessity for growth unless you have crazy virality but those are just the exception. Startups must spend money to make money so when you have an expensive necessity, you tend to be on top of it.

The illusion of control comes into play when the tools you use to measure marketing efficiency appear to be more robust and accurate than they truly are.

Enter “In platform” measurement.

Both Meta and Google freely give away robust measurement tools that help you understand what’s happening to your spend. “Oh wow! How kind of you Google!”

Now, why would two trillion dollar companies give away anything for free? WHAT?! They have ulterior motives. The horror.

Yes, in platform attribution is designed to make you spend more. It’s literally a casino. No windows (black box models). Free spins (give us your budget and we’ll do the optimization). Free drinks.

And now that Apple has become more annoying about measurement, companies have introduced proprietary probabilistic models that help you overcome them. Don’t get me wrong… these tools can be useful but it’s important to understand the business goal behind them. Just like you A/B test changes, so are Google and Meta on their “In Platform Measurement” to see what resonates more with you.

The illusion of certainty

Most early marketing channels are digital and so there’s an assumed certainty of being able to track the journey of a customer. This is not true and harder in 2025 than ever before. There’s a lot more privacy focus, competition, and non linearity. Because of this, what you’re trading off for “certainty” is completeness.

Let’s look at a journey that is fairly standard:

See an ad → ignore → friend talks about it → get interested → research → buy

In most situations, the in platform measurement might not give credit for the buy but turning it off would likely lessen the chance of the buy. Yes, word of mouth is powerful, but word of mouth when you’ve heard of something vs when you haven’t is very different.

Now, let’s run that same journey again but with one change:

See an ad → ignore → friend talks about it → get interested → research → see an ad → buy

In this second example, we feel that we have more “certainty” because the last click was an ad. This iillusion of certainty hurts most startups because they predominantly leverage digital channels.

How startups should measure Marketing

Goal

At an early stage, you need to answer one question above all else:

Are we growing efficiently?

That's it. Not "what's the ROI of TikTok ads versus Instagram?" or "how many people viewed our billboard?"—just "are we growing and is it sustainable?"

When you take this approach to measurement, you simplify what you measure and you spend more time optimizing the metric instead of debating the nuances of it.

Key Metrics

The best thing you can do is focus on a handful of metrics that actually tell you if your business is working.

• Blended CAC: Total marketing spend ÷ total new customers

• Payback period: How long it takes to recoup your CAC

• Retention rate: % of customers who stick around after 30/60/90 days

• Growth rate: MoM or QoQ revenue growth

Why blended CAC instead of platform-specific CAC? Because platform-reported metrics are:

• Biased (Meta will claim credit for everything)

• Unreliable (iOS 14.5+ ruined accurate attribution)

• Not comprehensive (they miss Word of Mouth and x-device conversions)

A real world example

You're spending $10,000 across Google, Meta, and TikTok, and you acquired 100 customers last month. Your blended CAC is $100.

But wait! Meta says they delivered 70 customers at a $71 CAC. Google says they delivered 50 at an $80 CAC. TikTok says they delivered 20 at a $100 CAC.

That's 140 customers out of 100. Welcome to the wonderful world of attribution overlap, where everyone takes credit for the same conversion.

For an early-stage startup, just use blended CAC. It's honest, simple, and actually useful.

Note: Whenever I suggest using CAC, the response is “But it doesn’t account for quality of acquistion". Yes, fair point BUT you can set your target CAC to whatever you want. No one is telling you your CAC has to be skyhigh. Should it be $5? $10? $500? I don’t know. Depends on your business.

Measurement Strategy

Your measurement needs to coexists with your marketing strategy. Early stage companies are still figuring out what works and where to invest so your measurement strategy has to be flexible to adapt but rigid enough to make decisions.

You can do this by prioritizing:

1. Quality over quantity of data

2. Quick analysis

3. Rapid experimentation

Most of your insights will come from the simplest analysis that you run on your big swings. Let’s say you introduce a new channel or make a large change to your creative strategy.

Did CAC go up or down significantly?

Are retention rates trending positively?

Which channel seems to be delivering the most customers?

Complement this with fast testing cycles where you iterate weekly but limit to one big variable per week. By limiting to one big variable, you can more easily measure the impact on your core metrics. Finally, document what you learned writing out why you decided to run this test, what metric you intended to move, and the result.

K.I.S.S. for tooling and testing

Tracking is not easy but it’s a necessity because it gives you slightly more information that you wouldn’t have had otherwise. Startups should prioritize UTM tracking (if they’re web based). If you’re mobile app based, it’s a lot harder but you’ll have to use something like AppsFlyer. Companies like AppsFlyer are known as MMPs (mobile measurement partner) and they’re the best you’ve got

For other analytics, tools like Amplitude give you free tiers as well and it’s a good place to start.

Most importantly, use what I’ll call common sense attribution:

• First touch for awareness channels

• Last touch for conversion channels

Then compare changes you make in channels to your blended CAC and make decisions using human judgment and pre /post tests

This list above isn’t definitive but it’s often what most startups use. If you’ve got a good reason to use first touch instead, then by all means go for it! But don’t go for MTA or an MMM right out of the gate.

Wrapping up

The key to startup marketing measurement is simplicity, speed, and actionability. You need enough data not perfect data.

1. Start with blended metrics

2. Prioritize speed over precision

3. Focus on unit economics

4. Keep your tech stack simple

5. Evolve as you scale

Enjoyed reading this?

Share with your colleagues or on your LinkedIn . It helps the newsletter tremendously and is much appreciated!

Missed my last article?

Here it is: How to beat ad fraud