This article was written by Mervi Sepp Rei, PhD - Director, Machine Learning at Zendesk, a partner of Digit 2024.
Data science is a big deal these days. It has the power to change how businesses operate, helping them make smarter decisions and stay ahead of the competition. But according to
Forbes, many companies aren’t seeing their data science teams reach their full potential. I’ve noticed this too in my own experience (though it sounds more convincing coming from Forbes).
Here’s why this happens — and what can be done about it.
Why data science teams don’t reach their full potential1. The problem isn’t clear or relevantOne major issue is when someone in the business decides, “Let’s have our data science team predict X,” without checking if “X” is even something that can be predicted or if it’s a problem worth solving. It’s like trying to answer a question that no one’s actually asking — it’s bound to fail.
2. Poor user experience (UX)Another common problem is that the tools and solutions developed by data science teams aren’t user-friendly. If the final product is hard to use or understand, customers won’t want to use it. A poor UX can seriously hold back the success of any data science project.
3. Data science and product don’t mix wellOften, data science teams work in isolation from product teams, creating a chasm between the two worlds. Product Managers (PMs) don't speak 'data science,' and data scientists don't speak 'product.' The prototypes that data scientists create and present to product teams are often very raw, making it unclear how to implement them in the current product offerings in a meaningful way. The value and shortcomings of these prototypes are not apparent to product teams, leading to further disconnect. Additionally, customers often find it challenging to understand how to use these features, and the feedback they provide rarely
reaches the data scientists. This lack of communication and integration leaves valuable insights untapped and underutilized, diminishing the overall effectiveness of both teams.
4. Users aren’t comfortable with the solution (yet)We’ve moved from a time when AI features were a novelty to a time when they’re expected. But here’s the catch — it can take a long time, sometimes more than six months, for people to get comfortable with AI-powered solutions. If users don’t trust or understand the AI, they won’t use it, no matter how good it is.
Good to know, but...
How can you fix it?My answer to all of these issues is: Let the data scientists take the lead, but make sure they’re accountable to the product team.
Fundamentally, data scientists should be solving customer problems — not building models/solutions that the product team prescribes. And for that they need to be close to the product and close to customers to get that understanding.
Users can be quick to reject AI that seems too confident or pushy, they need help understanding and trusting these systems. By guiding users through the probabilistic nature of AI — where mistakes can happen, but are often outweighed by correct decisions —
companies can better harness the power of their data science teams.
And don’t forget, inviting user feedback isn’t just good for customer satisfaction, it can also provide valuable insights to help improve and retrain your models.
Summing it upAt the end of the day, data science is all about solving real problems and creating value. But to get there, companies need to make sure their data science teams are set up for success. This means giving them the freedom to lead, but also holding them accountable.
To put it simply: When data scientists are empowered to lead with a clear problem, work closely with product and UX teams, and engage with users, they can help their companies achieve great things.
Mervi will also take the stage at the Digit Conference on October 4 with a keynote: "From Startup to AI Startup: NLP in the Age of LLMs".