Design and Evaluation of a Sound-Driven Robot Quiz System with Fair First-Responder Detection and Gamified Multimodal Feedback

Socially assistive robots are increasingly deployed in educational environments, yet many systems rely solely on verbal interactions, limiting engagement and fairness in group-based activities. This study presents the first sound-driven robot quiz system integrating multimodal feedback and Octalysis-based gamification to enhance user experience and fairness. The system uses non-verbal sound-based first responder detection (via cross-correlation), combined with verbal quiz answering, real-time gestures, music, and competitive rewards. A between-subject experiment was conducted with 32 university students comparing a verbal-only baseline against the proposed multimodal system. To evaluate user perceptions, we used subscales from the Technology Acceptance Model (TAM), Intrinsic Motivation Inventory (IMI), and Godspeed Questionnaire. We assessed students’ perceptions of perceived usefulness, ease of use, motivation (measured through the IMI subscales of enjoyment and competence), social presence (measured through the Godspeed subscales of likeability and anthropomorphism), and behavioral intention. Results show significantly higher scores (e.g., enjoyment d=3.11) across all measured dimensions for the multimodal group. The findings suggest that integrating fairness mechanisms and expressive feedback significantly increases acceptance and motivation in robot-assisted learning, providing a foundation for scalable, engaging educational HRI systems.