Hello experiMENTAList, it’s Sundar 👋

I’m a former Head of Marketing Science at Uber where I optimized $1Bn+ in spend across Brand, Performance, and Lifecycle. Now, I share weekly playbooks that help you prove and scale your Marketing ROI.

If you know a colleague or friend that might benefit from this newsletter, send them here.

Now let’s get experiMENTAL!

The DTC beauty category is crowded. To break through, Jennifer Anniston’s brand LolaVie, worked with Roku Ads Manager to easily set up, test, and optimize CTV ad creatives. The campaign helped drive a big lift in sales and customer growth, helping LolaVie break through in the crowded beauty category.

Learn more

Every week across the world, millions of people wake up and start their Monday mornings praying to the business gods. They hope that when they open their performance dashboard, their numbers have gone up and that they'll be able to share a 30-second blurb about this positive news in their weekly business reviews.

But as we all know, it's never that easy.

Today, I'll share the framework I use to triage business performance and save countless hours chasing rabbit holes.

If you get philosophical, business is nothing more than a facet of life, so the rules that govern your life also govern businesses. All forces can be broken down into three categories:

Most businesses get tripped up in #2. They think they have more control over those things.

Here are a few examples:

So to assess business performance, you need to systematically eliminate variables that affect your performance, helping you isolate where the impact is.

Where do I start?

With the things I can’t control. This might seem counter-intuitive, but it helps me make sure I'm not chasing the wrong rabbit holes.

The first thing I do is look at a year-over-year chart. You'd be surprised how often we forget how much seasonality it can impact things. Of course, we all remember large events like Black Friday, but what about minor holidays like President's Day or Indigenous Peoples Day?

Within holidays for example, there are three periods that surround the holiday:

An example from Uber:

The week before Thanksgiving was normal, then the week of Thanksgiving was obviously a huge drop, and then the week after Thanksgiving was a massive rebound. But even during the week of Thanksgiving, Thursday was obviously quiet, but the day before is one of the busiest days of the year. It's almost like a mini New Year's. Seasonality has so much nuance and even intra day intra hour seasonality needs to be considered.

Scheduled holidays are easy, but what about large events? Sports, concerts, festivals, etc.

An example from Bounce:

In Australia, which is one of our largest markets, we would have to continuously adjust because of the Taylor Swift effect. The year before, there was a set of Taylor Swift concerts that completely broke our numbers, and so the year after you needed to continuously think about why performance is lower.

How to triage:

Start by ruling out the things you can’t control because they often have a larger influence than you think and catch many people by surprise.

Influence vs. control are separated by a fine line. But, the things we can influence tend to be “higher up the funnel”. Naturally, the impact from there will waterfall down.

Example from Uber

We judged marketing performance by looking at how many first trips we drove. Here’s what our Marketing funnel looked like:

The experience in steps 2 to 5 was internally controlled, but the ad is a mix of your control and external influence.

So we'd start by assessing what metrcs like CPMs and CTR were relative to forecast.

If your CPMs are 10% higher than forecasted then keeping everything else constant you will have 10% lower performance (and conversely 10% higher CAC).

But we don't control CPMs.

Even if you set CPM caps and they're within a range, the truth is there is a volatility in that random metric that cascades and waterfalls down. Same thing with CTRs or ad conversion rates. Generally, they were within a range, but week to week there were fluctuations.

How to triage:

On 4., there's often a high probability that you've misforecasted. One way to offset that is to look at year-over-year growth in CPMs over the last few weeks and see if they've been consistent.

If you're consistently YoY 10% higher, then there's no reason this week would be any different. You need to make sure that's accounted for in your expectations.

This part of the triaging process is where alignment is crucial. There needs to be a consistent view of ownership and accountability for these metrics. Otherwise, every week it turns into a game of pointing fingers.

My advice here is to map out every metric and assign it an influence versus control label that you agree with your leadership team and your finance team.

There needs to be confidence we can can contain the metric, but understanding that these are random variables that are not fully controllable.

While this area might be the simplest, it's also the most controversial. If the product team introduces a bug and it brings down the whole site, is that really in our control? If the MarTech team breaks the product feed, is that really in our control?

It might sound existential, but in reality we do control these. Unfortunately, that also means the list of things we “control” grows quickly.

So, here’s a list of questions you need to answer to triage performance:

How to triage:

Here are some examples of subsegments to analyze every week:

To effectively triage business performance here you need to create systems. Otherwise, it’ll take you hours every week that’s just wasted on low ROI reporting. As an extension of that, if you want to read how to run a killer Weekly Business Review, check out my article here.

That’s it for this week!

Need a jam session to help you solve your Marketing Science challenge?Let’s chat >

Stay experiMENTAL,

Sundar