Whoever has will be given more? Exploring the Impact of Non-Linearity on Effect Heterogeneity in Psychological Research.

Non-linear relationships between an independent and a dependent variable are knownto inflate residual slope variance in multi-level models for observational data, when thenon-linearity is not specified in the analytical model. In a meta-analytical context, thesemodels are referred to as individual participant data meta-analyses. The inflation ofresidual slope variance is then interpreted as an artificial inflation of heterogeneity.Meta-analyses in the behavioral and psychological sciences often aggregateexperimental rather than observational data. These can also be acected by non-linearrelationships, but we cannot directly transfer techniques from observational studies,because experimental data lack continuous information on the independent variable.Here, we explore how non-linearity can acect experimental data and thus estimationsof meta-analytical heterogeneity. We introduce descriptors of heterogeneity thatindicate potential non-linearity and allow to identify regularities or patterns ofheterogeneous ecects that are not represented in conventional meta-analytical reports.Out of 56 initial multi-lab data-sets, we identify 20 with heterogeneous (unstandardized)ecects and report the descriptors, as well as plotted examples. We discuss if there areindications of non-linearity in the analyzed data and interpret plotted examples. Finally,we reflect the interpretability of individual participant data meta-analyses when appliedto data from multi-lab direct replication projects in psychological research.