Exercise: Characterizing Costs and Savings
Learn to characterize the costs and savings of a counseling program by selecting optimal thresholds for predicted probabilities, enabling successful business decision-making.
We'll cover the following...
Analyzing costs and savings of a counseling program
The connection between model output and business decisions the client will make comes down to selecting a threshold for the predicted probabilities. Therefore, in this exercise, we will characterize the expected costs of the counseling program, in terms of costs of offering individual counseling sessions, as well as the expected savings, in terms of prevented defaults, at a range of thresholds. There will be different costs and savings at each threshold, because each threshold is expected to result in a different number of positive predictions, as well as a different number of true positives within these. The first step is to create an array of potential thresholds. We will use 0 through 1, going by an increment of 0.01. Perform the following steps to complete the exercise:
Note: Additional steps to prepare data for this exercise, based on previous results in this chapter, have been added to the Jupyter Notebook. Please make sure you execute the prerequisite steps as presented in the Notebook before you perform this exercise.
- Create a range of thresholds, to calculate expected costs and benefits of
counseling, with this code:
This creates 101 linearly spaced points between 0 and 1, inclusive.thresholds = np.linspace(0, 1, 101)
Now, we need to know the potential savings of a prevented default. To calculate this precisely, we would need to know the next month’s monthly bill. However, the client has informed us that this will not be available at the time they need to create the list of account holders to be contacted. Therefore, in order to estimate the potential savings, we will use the most recent monthly bill.
We will use the testing data to create this analysis, as this provides a simulation of how ...