How to create an ROI scorecard for your machine learning product
Framework to show how your ML product generates financial value for your customer
Hello readers,
Welcome to Infinite Curiosity. I’ve been getting the following question a lot from ML practitioners who are at early stage startups — How do I prove that our ML product is creating financial value for customers? I humbly decided to tackle that question in this post. In the process, I used DALL-E to generate images for me based on what I wanted to write.
In this post, we’ll talk about:
What is ROI
Why do we need an ROI scorecard
Who needs to see it
Framework to create an ROI scorecard
If you have a question, submit it here and I’ll get back to you with my thoughts. Subscribe to this newsletter to receive it in your inbox every week:
I generated the above image using DALL-E. I wanted an image of a panda presenting the ROI chart to a customer. Looks like the system assumed that the customer is also a panda. Fair enough! Let’s dive in.
What is ROI?
ROI stands for Return On Investment. If you're selling a machine learning product to large companies, you need to find a way to show ROI. What this means is that if your customer spends $1 on your product, they need to gain more than that. Your product needs to create value that's higher than $1.
Why do we need an ROI scorecard?
Let's say you have a data intelligence product that helps your customers monitor their servers and applications. It detects anomalous behavior and surfaces it at the right time. That should be sufficient by itself, right? But it doesn't work like when you sell to large companies.
They need to prove internally that for every $1 they spend on this product, they're gaining more than that in value. A good rule of thumb is that the ROI usually has to be around 4x. It means that if the customer spends $1 on your product, they should realize $4 in business value that gets created. But how do we prove that to the customer? That's where the ROI scorecard comes in hand.
Another key benefit of doing the ROI scorecard is it helps you discover the pricing. Let's say you're selling your product for $10,000. But when you do this exercise, you realize that it's creating $120,000 in value for your customer. This is your cue to increase the price of your product. On the other hand, you may realize that it's only creating $20,000 in value. That's your cue to either find a way to create more value or decrease the price of your product.
Who needs to see it?
When large companies buy software products, they need to justify it to their finance department and to their leadership. They need to show that it's an investment that's generating financial value for the company. For example:
If it's a product that's supposed to increase revenue by providing sales intelligence data, then the increased revenue should be higher than the cost of the product.
If it's a product that's supposed to decrease the cost of data storage, then the cost savings should be higher than the cost of the product.
If it's a product that's supposed to reduce the errors created by manual entries of accounting data, then the value of those reduced errors should be higher than the cost of the product.
I generated another image here. I asked DALL-E what it would look like if a panda were presenting it to two people:
Framework to create an ROI scorecard
You need to find a way to connect the usage of this product to business value creation. An ROI scorecard usually has 3 columns:
Benefit: It's the business value being created
Before: What it cost them before they started using your product
After: What it costs now after they started using your product
It's important to note that you need to assign dollar value to each benefit. Or it will be a moot point.
Let's consider the data intelligence product we discussed earlier. In this example, let's assume that it's being used by an ecommerce company. For an ecommerce company, it's very important that their website is fully functional at all times. It needs be fast, responsive, accurate, and reliable. Every part of it impacts the company. Shoppers are very unforgiving when it comes to website performance. The company needs to make sure it's on top of it at all times.
The list of benefits depends on the product at hand. There are no specific rules here. You need to get creative.
In the above example, here are a few benefits you can list:
Latency: Is the website getting slower? Is that causing customers to abandon their shopping carts? If so, how much is the company losing because of that?
Alerts: Is a server failing? Are the right people being alerted as soon as it happens? What's the cost of this failure?
Resources needed to manage the process: How many people does the company need to manage their cloud infrastructure? How many people will they need after they start using your product?
Identifying root cause: How long does it take to detect the root cause of an incident? How much does that cost the company?
Detecting performance anomalies: How long before the company realizes that some part of their website is causing shoppers to drop off?
Predicting bottlenecks: More shoppers are coming to the company's website. Can it handle the increased traffic? Where are the bottlenecks?
All these items are critical to the ecommerce company. For each benefit listed above, you can assess how much it cost the company before they started using this product. And how much does it cost now after they started using your product.
You need to keep this ROI scorecard updated at all times. It takes some time for the benefits to accrue. You should plan to discuss it with your customer once every quarter to make sure you’re aligned.
If you are getting value from this newsletter, consider subscribing and sharing with your friends: