Multi-Interaction: A New Open-access Approachfor Studying and Comparing Multimodal Interactions Across Primates

Human language may have evolved as a tool of our complex interactional system. This hypothesis is supported by the results of studies showing that human and non-human primates share some interactional abilities. These comparative studies are sparse however, and their methodologies vary, most of them focusing on a single communication modality, which makes inter-specific comparisons difficult. We propose a new approach together with an open-access code (“Multi-interaction”) that aim at simplifying the study of multimodal interactions and can be applied for comparison purposes to many species, regardless of their communication repertoire. First, we choose to annotate all multimodal behaviors (i.e., communication signals and actions) that contribute to the informational content of the interaction and its overall structure. Second, the code we provide enables multimodal annotated interactions to be transcribed into unique sequences, using a method that preserves the following information: the units making up the interaction, their order of appearance, their emitter, their potential overlap (intra- and/or inter-individual) and their duration. Third, the code provides automatic computation of several quantitative measures that describe the overall structure of each interaction, such as overall duration, number of units, diversity of units, percentage of interindividual overlap, percentage of intra-individual superposition or each individual "presence" rate during the interaction. We aim to facilitate the processing of multimodal and multi-agent interaction data, enabling large datasets and species comparisons to be handled. By focusing on structure rather than meaning, we hope to promote human-decentered and species-general protocols in the study of interactions in primates, including humans.