Designing Co-located Social Interaction: The Role of Multimodal Media in Facilitating Sociability

Technologically-supported social interaction has gained significant attention within the Human-Computer Interaction community, particularly for facilitating remote social connections. However, less emphasis has been placed on co-located situations and multi-user scenarios where participants have unfamiliar relationships. We propose an augmented multimodal media method for co-located social interaction. Our study identifies three design principles for designing technologically-supported responsive environments: (1) Interaction over time, (2) Improvised social activity, and (3) Relational interaction. Grounded in these principles, we designed and evaluated four technology-mediated interventions in various social settings, including meal and open-ended play scenarios. Our findings demonstrate the impact of augmented sensory media on dynamic social cues, influencing bodily engagement, mutual eye contact, proximal sensitivity, and spontaneous conversation. We categorized a series of design insights as learning outcomes for researchers implementing augmented multimodal media approaches.