Propensity Score Estimation Using Neural Networks: A Comparison of DNN, CNN, and Logistic Regression

Background: Observational studies serve as an alternative to randomized experiments, but they can introduce selection bias. Propensity score (PS) methods address this issue by balancing covariates between groups. For the best estimation of PS, all possible and important covariates should be included in the estimation model. However, if too many covariates are included in the estimation model, problems may occur due to the high dimensionality. As the solution, many studies suggested neural networks for high-dimensional data. Methods: This study utilized a deep neural network(DNN) and a convolutional neural network (CNN) for PS estimation, which can algorithmically handle nonlinear relationships and interactions of covariates. A simulation was conducted to examine the performance of the DNN and CNN in comparison to logistic regression (LR). Results: This study demonstrated that DNN and CNN are feasible and effective as PS estimators, outperforming LR in reducing bias. Specifically, while LR increased bias in the outcome by 17% and could only decrease 5% of covariate imbalance, DNN and CNN were capable of reducing 13% and 16% of bias in the outcome respectively, alongside diminishing 16% and 21% of covariate imbalance. Conclusions: Although the application of neural networks requires specialized tools and additional system resources, they offer significant advantages when the precise estimation of treatment is essentially required with many covariates. However, we found that for reliable PS estimation using DNN and CNN, it is imperative to establish and adhere to proper error rate thresholds. Therefore, it is imperative to utilize these machine learning techniques for PS estimation judiciously until appropriate error rates are determined and methods for achieving these error rates are established.