Machine Learning Approaches for Meta-Analytic Estimates of Important Predictors in Behavioral Science Studies: An Analysis of Cooperation in Social Dilemmas

Research in the social and behavioral sciences is accumulating at an exponential rate. A challenge facing scientists is to use the accumulative research to quantitatively determine which predictors contribute the most to explaining variation in a specific phenomenon. Here, we provide a novel application of machine learning to meta-analyze an entire body of research on cooperation in social dilemmas. We utilized the Cooperation Databank, a machine-readable dataset of 2,636 experimental studies on cooperation (1958-2017), to conduct a meta-regression to assess the relative importance of 56 predictors of cooperation, including parameters of the experimental paradigm (e.g., group size, payoff structure, and repeated interaction) and sample characteristics (e.g., gender, age, and ethnicity). We used state-of-the-art ensemble machine learning regression models and applied the grouped permutation feature importance method for importance estimation to achieve robust and accurate estimations. The analysis revealed that the top three most important predictors were expectations of partner behavior, preferences for conditional cooperation, and punishment. We also found that the important rankings of the predictors varied by game type, with the prisoner's dilemma and public goods dilemma having a high rank correlation (rs = .804), while the resource dilemma game had lower correlations with the other games (rs = .444). This quantitative method can yield valuable insights about how to improve cooperation within different social dilemmas. The analytical methods developed in this study can be applied across the social and behavioral sciences, especially in well-developed topics that involve accumulated empirical studies.