Repetitive Research: A Conceptual Space and Terminology of Replication, Reproduction, Re-Implementation, Re-Analysis, and Re-Use in Computational Literary Studies

This article is motivated by the ‘reproducibility crisis’ that is being discussed intensely in fields such as Psychology or Biology but is also becoming increasingly relevant to Natural Language Processing and Digital Humanities. Using the phrase ‘repetitive research’ as an umbrella term, and with the objective to provide clarity and help establish best practices in this area, this article focuses on two issues: First, the conceptual space of repetitive research is described across five key dimensions, namely those of the research question or hypothesis, the dataset, the method of analysis, the team involved, and the results. Second, building on this new description of the conceptual space and on earlier terminological work, an easily understandable terminology for recurring scenarios of repetitive research is proposed. For each scenario, its typical purpose and added value in the research process are discussed, the requirements for enabling it are described, and it is illustrated using examples from the domain of Computational Literary Studies. The key contribution of this article, therefore, is a proposal for a transparent terminology underpinned by a systematic model of the conceptual space of repetitive research.