Set and Don’t Forget

It’s no longer rare to see machine learning (ML) models being used to support a variety of business decisions, from whether a medical claim should be paid or sent to the fraud investigation team, to what route will be more efficient for a delivery truck or what discount should be offered to a distributor.

But while these ML-based solutions can be powerful tools to improve core operations, a significant number of organizations still consider the job done when the model is installed and running, unaware of the risks that this “set and forget” approach creates for the business.

In practice, without robust post-deployment monitoring, the gains extracted from predictive models are likely to be short-lived.

To understand why that happens, imagine a company using an ML model to recommend personalized retention offers to individual customers. The model goes through a rigorous test of accuracy that shows an increase from 75% to 90% in the customer retention rate. The new model is deployed, the promised results are achieved, and everybody is happy.

Click