Preprocessing and analysis practices in developmental N400 research – a systematic review and pipeline comparison

Electroencephalography (EEG) data have to undergo a complex preprocessing and analysis pipeline before inferences about patterns of brain activity timelocked to critical stimulus events (event-related potentials) can be made. In order to set up this pipeline, ERP researchers have to make many decisions, introducing analytic flexibility and researcher degrees of freedom. Recent studies have examined this flexibility, its consequences and ways to reduce it in the adult ERP literature. However, these problems also apply – and we as suggest, even to a greater extent – to infants’ and children’s ERP data, which has received little attention with regard to analytic flexibility so far. To address this problem, we conducted a systematic review to assess common practices in infant ERP preprocessing and analysis steps. We focused on papers investigating the well-studied N400 component in one of its most common applications, word learning, in pre-school children (0-5 years). We identified 31 papers using a PubMed literature search. We analyzed 47 practices of these studies, including properties of preprocessing steps and statistical analysis. For each of these practices, we investigated their implementation and reporting. Worryingly, we found that each study used a unique preprocessing and analysis pipeline. Individual practices differed greatly in how they were implemented and reported, with some practices being reported perfectly according to commonly accepted reporting guidelines, and others not at all. Based on the results of this systematic review, we compared a typical preprocessing pipeline used in the field with a recent standardized pipeline specifically developed for developmental ERP data. We document that the standardized pipeline results in both better data retention and data quality. Against these findings, we discuss what infant ERP researchers can do to increase reporting and consistency in preprocessing and analysis steps, given the unique challenges of infant data. We highlight that (1) infant ERP studies can only be reproducible when preprocessing and analysis steps are consistently reported, (2) the many decisions researchers have to make during preprocessing and analysis can influence the results, and (3) we should strive towards standardized preprocessing and analysis pipelines to make results comparable.