Issues in Solving the Problem of Effect Size Heterogeneity in Meta-Analytic Structural Equation Modeling: A Commentary and Simulation Study on Yu, Downes, Carter, and O’Boyle (2016)

This paper is in press at Journal of Applied Psychology.Abstract:Meta-analytic structural equation modeling (MASEM) is becoming increasingly popular for testing theoretical models from a pool of correlation matrices in management and organizational studies. One limitation of the conventional MASEM approaches is that the proposed structural equation models are only tested on the average correlation matrix. It remains unclear how far the proposed models can be generalized to other populations when the correlation matrices are heterogeneous. Recently, Yu, Downes, Carter, and O’Boyle (2016) proposed a full information MASEM approach to address this limitation by fitting structural equation models from the correlation matrices generated from a parametric bootstrap. However, their approach suffers from several conceptual issues and technical errors. In this study, we reran some of the simulations in Yu et al. by correcting all of the errors in their original studies. The findings showed that bootstrap credible intervals (CVs) work reasonably well, while test statistics and goodness-of-fit indices do not. We advise researchers on what they can and cannot achieve by applying the full information MASEM approach. We recommend fitting MASEM with the TSSEM approach, which works well for the simulation studies. If researchers want to inspect the heterogeneity of the parameters, they may use the bootstrap CVs from the full information MASEM approach. All of these analyses were implemented in the open-source R statistical platform; researchers can easily apply and verify the findings. This paper concludes with several future directions to address the issue of heterogeneity in MASEM.