Adaptive Particle Swarm Optimized XGBoost Ensemble Algorithm for Online Credit Scoring

Abstract Determining the credit worthiness of an individual for a loan approval is a challenging task for most financial institutions. To assess the credibility of an individual, a credit score model is often used and it encompasses factors such payment consistency of customers and their behaviors and their previous behaviors. Due to its easy interpretation and outstanding prediction performance for both static and time-varying, evolving domains, machine learning has gained a lot of reputation over statistical approaches. For this reason, machine learning has gained much attention in the automation of the design of effective and efficient credit scoring techniques over the years. Credit scoring task is also an ephemeral scenario as most of the variables are likely to drift overtime, making the use of stream mining suitable for detecting and adapting to changes in the underlying data distribution. Given a problem in such time-varying and evolving environment, credit scoring models that were once robust may become suboptimal if the behavior of customers change over time as their earnings get eroded by changes in interest rates, natural disasters and increases in pay as you earn, rendering the current credit score model outdated. In credit scoring, many of the variables tend to drift over time resulting in an ephemeral scenario that requires online machine learning algorithms tailored for learning in time-varying and evolving environments where the target concept changes. Since machine learning algorithms can learn incrementally, they are able to detect learn changes that occur in the data. In this paper, we propose an online Decision Trees based eXtreme Gradient Boosting (XGBoost) credit scoring model that is based on a heterogeneous adaptive particle swarm optimization where behaviors change at each iteration by randomly selecting new behaviors from the behavior pool. The prediction performance of our approach is evaluated first as a batch learning algorithm and secondly as a data steam learning algorithm using a variety of validation schemes for traditional batch learning and the Kolmogorov-Smirnov and Population Stability index metrics. Each behavior consists of a pair containing a position update and a velocity update. Decision Trees can easily sift through credit data characterized by a high dimensional curse and a complex correlation. The behavior of our proposed approach is evaluated for both static and dynamic domains since customer variables tend to drift over time. Empirical experiments conducted showed that our proposed approach performs comparatively well in both static and dynamic environments on four datasets.