Credit Risk Modeling Using Transfer Learning and Domain Adaptation

Abstract

In the domain of credit risk assessment lenders may have limited or no data on the historical lending outcomes of credit applicants. Typically this disproportionately affects Micro, Small, and Medium Enterprises (MSMEs), for which credit may be restricted or too costly, due to the difficulty of predicting the Probability of Default (PD). However, if data from other related credit risk domains is available Transfer Learning may be applied to successfully train models, e.g., from the credit card lending and debt consolidation (CD) domains to predict in the small business lending domain. In this article, we report successful results from an approach using transfer learning to predict the probability of default based on the novel concept of Progressive Shift Contribution (PSC) from source to target domain. Toward real-world application by lenders of this approach, we further address two key questions. The first is to explain transfer learning models, and the second is to adjust features when the source and target domains differ. To address the first question, we apply Shapley values to investigate how and why transfer learning improves model accuracy, and also propose and test a domain adaptation approach to address the second. These results show that adaptation improves model accuracy in addition to the improvement from transfer learning. We extend this by proposing and testing a combined strategy of feature selection and adaptation to convert values of source domain features to better approximate values of target domain features. Our approach includes a strategy to choose features for adaptation and an algorithm to adapt the values of these features. In this setting, transfer learning appears to improve model accuracy by increasing the contribution of less predictive features. Although the percentage improvements are small, such improvements in real world lending could be of significant economic importance.

Integrating Machine Learning in Anti-Money Laundering through Crypto: A Comprehensive Performance Review

“…Transfer learning and domain adaptation are techniques used in machine learning to apply knowledge from one domain or dataset to another, improving the efficiency of models in detecting financial crimes in the context of Anti-Money Laundering (AML) (Suryanto et al, 2022). Transfer learning involves transferring knowledge from a source domain, where there is an abundance of labeled data, to a target domain, where labeled data may be scarce.…”

“…Domain adaptation is a related concept that focuses on adapting a model trained on one domain to perform well on a different, but related, domain. In AML, domain adaptation can be used to adapt models trained on data from one financial institution to perform well on data from another institution, even if the data distributions are slightly different (Suryanto et al, 2022). Both transfer learning and domain adaptation can improve the efficiency of machine learning models in AML by reducing the amount of labeled data needed for training and by leveraging knowledge from related domains or datasets.…”

How Far Has Our MSMEs Credit Underwriting Assessment in Indonesian Commercial Banks Progressed?

“…Advanced analytics also enables building a model from an existing model, as known as transfer learning, which could improve model accuracy and allow whoever with low data availability to reuse or transfer information from previously learned tasks. For example, a study by Suryanto et al (2022) used the data from peer-to-peer lending (Lending Club datasets) to predict the probability of default of small businesses from a debt consolidation model. They used the Progressive Shifted Contributions method to varying network architecture and hyperparameters to find the best balance of learning from source and target domains and investigate how and why transfer learning improves model accuracy using Shapley Additive exPlanations values.…”