An Online Brain-Computer Interface for Detecting Incongruity in Augmented Reality Applications

Objective Augmented reality can provide digital information about physical entities presented within its real-world context. However, this information might disagree with the user’s expectations due to factual errors in the data or cognitive biases. Such incongruity can impair user experience and undermine trust in the AR system. To address this issue, we propose detecting inconsistencies between physical objects and digital information through hybrid brain-computer interfaces. Approach We conducted two complementary experiments. First, we implemented a strategy that integrates eye-tracking and brain signals for incongruity detection in an offline study. Subsequently, we assessed our approach in an online study in which participants received immediate feedback on the classification. Main results The grand average event-related potentials revealed consistent electroencephalographic responses to incongruent augmentations, specifically a centroparietal N400 effect, across both experiments. We could further distinguish between congruent and incongruent information with an average balanced accuracy of 70 % in the online study. Significance These findings demonstrate the feasibility of detecting incongruity online, allowing for autonomous system adaptation, like presenting information in a more accessible format or providing contextual support.