How to create a culture of experimentation

👋 Hey, it’s Sundar! Thanks for reading experiMENTAL: my newsletter that helps founders navigate the CRAZY world of consumer tech with secrets from 10+ years in growth at Uber & others.

In this week’s newsletter, you’ll learn:

1. what a culture of experimentation is and why it’s necessary for hyper growth

2. how Booking.com’s culture of experimentation led to disrupting the travel industry

3. how you can install that culture and speed up your growth

What is a culture of experimentation?

The phrase “culture of experimentation” sounds like a buzzword made up by a McKinsey consultant (who was likely paid an obscene amount).

But, it’s the foundation for every successful growth tech company.

So, what is it?

It’s a manifestation of 2 simple principles:

1.Evidence is more important than hypotheses

Decisions at companies are made under the assumption that we know what’s best for our customers.

And then they roll out that feature and nothing happens 🤯.

Instead, you need to go from “This WILL work” → “This MIGHT work”

You let a test dictate what happens instead of forcing a change.

2.Data is more important than seniority 

Everyone has experienced being trampled by a HiPPO.

HiPPO stands for “Highest Paid Person's Opinion” and it’s when the highest paid person's opinion carries more weight than anybody else's in the room.

At a company with a culture of experimentation, HiPPOs should rarely exist.

If the most junior person has data that disproves the CEO’s opinion then there’s only 1 winner: The person with the data.

Why a culture of experimentation is important

A culture of experimentation is good for the entire business because it:

• Empowers employees

• Promotes innovation

• Simplifies decisions

• Mitigates risks

Employees stay longer and are happier.

Growth finds new channels and new opportunities.

Decisions are more black and white.

Actions have known benefits and risks so they’re more informed.

It’s a better way to build a business and often leads to better long term outcomes.

If you’re still not convinced, think about companies that do the opposite:

Blockbuster is a great example of a company that didn’t have a culture of experimentation.

They were stagnant stagnant, boring, and definitely not innovative.

Now, they’re extinct.

Don’t be like Blockbuster.

Examples of awesome cultures of experimentation

What’s the common thread between Meta, Google, Netflix, Uber, Amazon, Booking, AirBNB?

1. Product market fit

2. Culture of experimentation

Here’s where these companies started

Facebook → college social networking

Google → search

Netflix → DVD rentals

Uber → black cab

Amazon → books

Now think about where those companies are.

They found product market fit in their original niche.

And then they experimented.

Let’s go a bit deeper into an example.

There’s a famous Booking.com story that truly encapsulates their best-in-class culture of experimentation.

It’s December 2017.

They’re prepping for the busiest holiday travel period.

Enter Director of Design.

Gillian Tans (CEO at the time) is nervous. Her palms are sweaty. Knees weak and arms are heavy.

Lukas Vermeer (Head of Experimentation) bets a bottle of champagne that the test will flop.

What happened?

Well here’s Booking’s current home page so see for yourself:

Their current home page 7 years later is effectively what the Director of Design suggested.

If all of the management was opposed to it, why did it happen?

1. Evidence is more important than hypotheses

2. Data is more important than seniority

Booking.com has a core tenet that “Anyone at the company can test anything—without management’s permission.”

Just think about how powerful that is.

That’s culture.

How to build a culture of experimentation

Experimentation is a human function. We’ve been doing it since Day 1.

And it’s no different within a tech company.

It doesn’t have to be complicated.

Now, forewarning that building a culture of experimentation is not easy at all.

My goal is to demystify it a bit hoping that people start implementing one without feeling overwhelmed.

First, you have to realize that everyone will be impacted by a culture of experimentation.

It might look different within each group but it must be a collective effort.

The second step is to think of experimentation as a continuous cyclical funnel

Wtf is a continuous cyclical funnel?!

Continuous

Experimentation can’t stop when you want it to and turn it on when you want it to.

The culture must be always on.

Cyclical

Experimentation goes through cycles of research → experiment → roll out → research.

If there’s a break in the cycle then you’ll often find experimentation stall. The challenge is keeping the flywheel and culture going.

Funnel

Experimentation requires 7 steps:

1. Research

2. Hypotheses

3. Ideas

4. Features

5. Experiment

6. Analyze

7. Rollout

If you don’t adhere to this general order then you’ll introduce biases and undermine the culture of experimentation.

Here are some best practices I’ve learned over 10+ years of experimenting that will help you make the best out of each step.

I could write another 10 pages around this but I know y’all don’t want to read that, so reach out to me if you have any questions on the things I mentioned above.

And that’s it for this week. I hope you now know a bit more about:

• what a culture of experimentation is and why it’s necessary

• Booking.com’s culture of experimentation

• how you can start installing that culture

Good luck to you and start experimenting!

Thoughts? Questions? Feedback? Let me know.

If you found this useful and want to work more together, I help consumer tech founders with 1 of 2 programs:

Pre PMF: MAXIMIZE RUNWAY

• 4 workshops that review your customer journeys, growth model, data foundations, and weekly rituals.

• Advisory work where I quarterback your weekly business reviews helping you prioritize, ship, experiment, and find PMF quicker without burning runway.

Post PMF: SCALE PROFITS

• 4 week audit of your current priorities and processes scoring you on how data driven your company is.

• Growth Analytics leadership where I install growth frameworks we used at Uber to help your teams identify, acquire, and retain high value customers. No more wasted spend on customers you'll never see an ROI on.

Book a discovery call to find out more.