By: Tracy Bremmer
In our last blog (July 30), we covered the first three stages of model development which are necessary whether developing a custom or generic model. We will now discuss the next three stages, beginning with the “baking” stage: scorecard development.
Scorecard development begins as segmentation analysis is taking place and any reject inference (if needed) is put into place. Considerations for scorecard development are whether the model will be binned (divides predictive attributes into intervals) or continuous (variable is modeled in its entirety), how to account for missing values (or “false zeros”), how to evaluate the validation sample (hold-out sample vs. an out-of-time sample), avoidance of over-fitting the model, and finally what statistics will be used to measure scorecard performance (KS, Gini coefficient, divergence, etc.).
Many times lenders assume that once the scorecard is developed, the work is done. However, the remaining two steps are critical to development and application of a predictive model: implementation/documentation and scorecard monitoring. Neglecting these two steps is like baking a cake but never taking a bite to make sure it tastes good.
Implementation and documentation is the last stage in developing a model that can be put to use for enhanced decisioning. Where the model will be implemented will determine the timeliness and complexity for when the models can be put into practice. Models can be developed in an in-house system, a third-party processor, a credit reporting agency, etc. Accurate documentation outlining the specifications of the model will be critical for successful implementation and model audits.
Scorecard monitoring will need to be put into place once the model is developed, implemented and put into use. Scorecard monitoring evaluates population stability, scorecard performance, and decision management to ensure that the model is performing as expected over the course of time. If at any time there are variations based on initial expectations, then scorecard monitoring allows for immediate modifications to strategies.
With all the right ingredients, the right approach, and the checks and balances in place, your model development process has the potential to come out “just right!”
