Application of tree based enhanced stacking ensemble learning

Since it is difficult for machine learning algorithms, such as XGBoost, LightGBM, to extract feature interaction effectively and deep learning algorithms have high time complexity, a tree enhanced stacking integrated model is proposed in this paper, based on tree model feature generation theory and convolutional neural network(CNN) feature extraction theory. Firstly, to effectively extract feature interaction, tree-based-enhanced-stacking generates new features through tree model LightGBM and merges them with the original features. Secondly, to enhance the presentation ability, LightGBM is modified into ML LightGBM with hierarchical structure. Finally, ML_LightGBM, XGBoost, Random Forest and other models are introduced into the stacking integration framework to generate secondary data sets. To enhance the deep feature extraction ability of tree-based-enhanced-stacking, CNN model is used to extract features from the secondary data sets. Compared with ML_LightGBM, XGBoost, Random Forest in two data sets ppdai and default, the results show that the tree enhanced stacking model proposed in this paper has significantly improved in AUC, ACC, KS, loss and other indicators.