- experiMENTAL
- Posts
- How to master Pre / Post analyses
How to master Pre / Post analyses
It’s the least complex and least accurate method, but the most commonly used. Here’s how to effectively use it.
Sundar Swaminathan
March 12, 2025
👋 Hey, it’s Sundar! Welcome to a paid subscriber only article of experiMENTAL: a weekly newsletter on B2C Marketing & data science how-to guides, frameworks, and stories from 15 years including early Uber. Free tier subscribers can continue reading for limited preview of the article.
How to master Pre / Post analyses
If you tell any data scientist that you used a Pre / Post analysis to measure impact, our gut reaction will be to laugh at you. There’s actually a class we all take “How to laugh at Marketers 101” and it’s the first lesson. But, as I’ve had more time in my career, I realize that saying Pre / Post is bad is an elitist view.
The alternatives to Pre / Post like A / B testing require things that not everyone has access to:
Resourcing to set up tests
Culture of experimentation
Skillset to analyze it
Patience / time
It’s things many of us take for granted when we’re lucky to be at companies like Uber, Netflix, Spotify etc… but we’re in the .1%. F*ck that. My mission has always been to help Marketers make smarter decisions. Nowhere in my mission statement does it say “ I only want to help those that can afford it”.
So, here’s how you can master the Pre / Post analysis.
The pitfalls of Pre / Post
Confucius said “To understand how to measure Marketing, you must understand how not to”. I wrote an article called “The 5 ways to measure Marketing ROI” and Pre / Post is the least accurate and least complex methodology available.
Here’s what I said about Pre / Post in that article:
The Pre/Post methodology is named that because it compares a metric from the pre period of a campaign (Pre) to what happens in the post period (Post).
It’s a simple method that has been around since the dawn of time.
The biggest challenge is that it’s very hard to isolate the impact to your metric of interest from just your campaign.
In today’s world there are so many factors:
PR
Seasonality (the ultimate scapegoat)
Competition
Pricing changes
Product changes
Algorithm changes
Macro factors like war, economy, etc.
With the emergence of more digital channels and social media, your campaign and company can explode or implode within days.
You might think ”I’ll keep looking back longer in the Pre period to stabilize the data set”.
You’ll end up just spending more time explaining through factors.
It’s just really tough and when you present the analysis people from random teams will chirp in with “Have you thought about this”. You couldn’t have thought of everything.
All of this is true.
But, what if you have no choice?
Reply