How Data Scientist should use AI

👋 Hey, it’s Sundar! Welcome to a 🔐 paid 🔐 article of experiMENTAL: a weekly newsletter on B2C Marketing & data science how-to guides, frameworks, and stories from 15 years including early Uber.

This post is brought to you by Roku

CTV Advertising Unlocked

Roku Ads Manager makes TV streaming advertising simple.

You can:
• Start with just $500
• Optimize in real time
• Create interactive experiences with Action Ads

Get your brand on the big screen today and reach engaged viewers on America’s #1 platform.

*By hours streamed, Dec 2023
(Hypothesis Group)

Reach more customers on CTV.

How Data Scientist should use AI

In a world where everyone thinks the answer to everything is AI, the truth is not even close to that. Every week, I chat / work with data scientist and the reality is that AI is a disruptive technology but it’s really more of an enhancing one than something that will replace the field (at least in its current state).

To explain this, I’m going to borrow a concept from finance called Alpha and Beta.

Example: You buy Apple shares and it’s up 25% this year.

Beta → The beta of a portfolio is how much of your returns are related to the broader market. Example : The S&P 500 is up 10% this year. That would generally be considered “the market” and so that’s the Beta

Alpha → The alpha of a portfolio is how much of your returns are related to your decisions. You chose to only buy Apple instead of the broader market and so you’ve created a 15% Alpha.

10% (Beta) + 15% (Alpha) = 25%.

So, how does this work with Data Science? The Beta will essentially be the expected and the average performance. The Alpha is what you can layer on top to create better performance than your average peer. Using this concept, we’re going to explore how Data Scientist should use AI by breaking down the core work flow of a data scientist (more detail on the core work flow here) and how you can find Alpha and Beta.

Core work flow:

1. Prioritize

2. Analyze

3. Communicate

4. Execute

Prioritize

The first step for every data professional should be learning how to prioritize their work which is a nice way of saying “learn how to say no”.

The reason prioritization is challenging is for a few reasons:

1. Seniority

   Most data professionals come in quite junior and there’s a lack of skilled data management which means that data folks are put in the position of having to say no to more senior leaders which is a natural power imbalance

2. Context

   Data folks often have incomplete information which means that making fully baked prioritization is difficult and often impossible. Also to prioritize, you need something to prioritize against whether it’s a KPI or a Strategy and let’s be honest about how bad companies are at this

3. Interest

   The last reason prioritization is hard is because no one is really interested in prioritization. Most data folks are naturally curious and so it’s easy to say yes when there’s so much potential to analyze and find cool insighst (spoiler… you’re job is more than that)

So, let’s now revisit the Alpha + Beta concept.

If you’re below the Beta (aka below the average) at prioritization then AI can kinda somewhat sorta help you. I mean that to be intentionally vague because AI has been trained on enough data to sense what may feel like a bigger priority that it should be able to tell you on average what is worth prioritizing.

However, it doesn’t necessarily have context on your company and leadership dynamics. For example, when you’re founder comes in and says “Hey, I need you to work on XYZ by this weekend for an investor deck”, the reality is that it’s not objectively a priority. It’s emotionally a priority.

Gen AI also doesn’t have the dynamics of understanding your relationship with the person asking / determining the priority. There have been many instances where I’d feel comfortable saying “No, we don’t need this now” and other times where I haven’t.

Within the Beta + Alpha framework:

Beta (average) → Use Gen AI to help you prioritize knowing that it’s taking the average and assuming the priorities. It can also help you with exercises like T-Shirt sizing if you provide it the right context.

Alpha (beat the average) → The best way to do this is to shift to a more human approach. Make sure you have a seat at the table and are aligning with stakeholders as partners and not as a subservient query monkey. This is where the best data scientist separate themselves which AI can’t do yet.

Summary: Good at getting to Beta but not for finding Alpha

Analyze

The analysis step is where Gen AI has the most advantage for a data scientist / analyst.

Breaking down an analysis into a scoring rubrik:

• Speed

• Quality

• Completeness

In all 3 steps, you can leverage AI to significantly improve (in the realm of at least 20%).

Speed → This is the most obvious one that EVERY data professional should be using Gen AI for. In 2025, there is no reason you should be handwriting SQL at all. I’ve tried this out multiple times and it has saved me hours every time. It’s not perfect because your prompt is likely not perfect but there are other times where it just writes the wrong SQL (again can be fixed with better prompting) . Speed for a data scientist / analyst is so important because it allows you to do more analyses which therefore creates more impact and gives you bigger opportunities and so the cycle continues.

Get access to the rest of this article...