A Taxonomy of Data Synthesis: A Tutorial

As more data is shared and concerns over the replicability, reproducibility, and generalizability of psychological and other social sciences continue, more researchers aim to conduct multi-study or multi-sample research and synthesize findings via data synthesis, using different parameterizations of individual participant meta-analysis. However, there is no overarching framework organizing different parameterizations and a relatively small number of simulation-based or empirical examples testing or comparing these parameterizations. Thus, this tutorial paper has three main goals. First, we provide an overview of six parameterizations of individual participant meta-analysis, which we organize into a taxonomy based on different features of each parameterization (e.g., sample-specific parameters, meta-analytic parameters, number of models required). Second, using empirical data from 26,205 participants across 11 longitudinal studies, we provide a tutorial estimating each parameterization by investigating prospective meta-analytic and sample-specific associations between the Big Five personality traits and crystallized abilities along with four moderators of these associations. Finally, we compare convergence and divergence of findings across methods. We found that Openness is a robust predictor of crystallized abilities across samples and methods and that there were few moderators of personality trait-crystallized ability associations. Across methods, we largely see convergence in model estimates, with some exceptions. We conclude by making recommendations and providing a flow chart for choosing the most appropriate parameterization of data synthesis given a particular team’s research goals, questions, data availability, and model features.