Top data science business metrics

Introduction

Many data science or algorithm metrics are useful to know, such as MAE and RMSE, but there are other metrics that can be more meaningful to stakeholders and your organization as a whole. It is equally vital to comprehend, practice, and put into action, despite the fact that it is not as well-taught in academic contexts.

I'm going to look at three instances in which using business analytics to interact with stakeholders can be advantageous in the future (especially the ones who are not in data science).

Bucketed Unit Ranges

 

Always keep in mind that not all of these metrics will be applicable to all use cases, which is why I'm offering an example of one so you can determine whether it's the correct fit for your situation. Keeping this in mind, let's consider a case in which you're anticipating a continuous aim, such as a range of numbers ranging from one to hundred.

 

You can use MAE or RMSE, for example, to figure out how well your unique model is performing in the present market environment. When engaging with executives or other stakeholders, however, it is important to describe the issue in terms of how it affects the company's overall financial performance.

 

Predicting the amount of money a house will be worth in order to put it on the market is an example of a practical application of machine learning. Anywhere between $200,000 and $300,000 would be a respectable goal to strive for.

 

Notice that your MAE is $10,000 and your RMSE is $30,000; these are your margins of error. After you've specified your features and developed your model, you'll discover that your MAE is $10,000 and your RMSE is $30,000.

 

This is the value of your MAE and RMSE combined. It is possible that your stakeholders will not find this information useful in making future business decisions, despite the fact that it appears amazing and beneficial to your model.

 

Instead, you might use the business key performance indicators (KPIs), which are essentially bucketed dollar amount ranges, as follows:

 

●     For example, the percentage of predictions that were within $10,000 of the actual result: Approximately 40% of the projections were within $10,000 of the fact. To examine this business metric in a different way, we may look at how many of our predictions were underestimating in relation to the actual percent of predictions that were $10,000 greater than the actual.

 

●     For example, 20 percent of estimates were within $10,000 of the actual amount paid in cash. If we look at this business metric from another perspective, we can observe how many of our predictions were incorrect when compared to the actual outcomes.

 

This business measure technique has the advantage that stakeholders and you will both be able to see how far off the mark you are on either side of the projection. For example, the MAE can only look at estimations that are both below and above the real condition (as represented by the mean absolute error). Depending on your use case, you may want forecasts to naturally overstate or underestimate the true value.

 

Change in Impact

Having gained an understanding of the various ways in which a mistake might be seen, we can now analyse the consequences of the preceding metric for the firm.

 

Consider the implications of looking at the number of predictions in a bucketed range of money. For example, we might wish to consider what it means for the company.

In this section, we will look at a variety of examples and conversations that have taken place around how impact business measurements have changed through time:

 

●     Overestimating reduces revenues

Ex: how many purchases take longer than two weeks when we overestimate?

 

●     Sellers lose money when they undervalue.

ex: what percentage of offers were undervalued?

 

Although this use case is centered on real estate transactions, it may be used to a variety of other circumstances as well, as you can see in the example. Take, for example, automotive sales, product sales, and so on. Not all metrics must be sales-related.

For example, how many users registered into your company's app after you were shown a set of forecasts could be a simple measure of engagement.

 

A/B testing is commonly used to determine the impact of your forecasts. This is a comparison of what happens when you make specific predictions and which areas of your business are most important to you or your stakeholders, as well as whether or not those metrics have changed considerably.

 

Conclusion

For the purposes of this discussion, we will use the terms MAE and MAPE to refer to two different types of business metrics. As we can see, simply discussing the MAE, MAPE, AUC, and other metrics is not adequate when adopting a model into an official release for a firm.

 

When testing a model in terms of the business, it is always advisable to test it in terms of the business. We have covered how bucketed metrics, as well as their impact on product and user behavior, can help us understand the business better.