Group interaction field for learning and explaining pedestrian anticipation

Abstract Anticipating others’ actions is innate and essential for humans to navigate and interact well with others in dense crowds. Such ability is urgently demanded by unmanned systems, such as service robots, self-driving cars, etc. However, existing solutions hardly predict pedestrian anticipation accurately because the influence of group-related social behaviours has not been well considered. While the group and group interactions are ubiquitous and significantly influence pedestrian anticipation, this influence is diverse and subtle, making it difficult to explicitly quantify. We innovate the group interaction field (GIF), a novel group-aware representation that quantifies the pedestrian anticipation into the probability field of pedestrians' future locations and attention orientations. An end-to-end neural network, GIFNet, is proposed to estimate the GIF from explicit multi-dimensional observations. GIFNet quantifies group behaviours' influence by formulating a group interaction graph with propagation and attention adaptive to group size and dynamic interaction states. The experimental results show that the GIF can effectively represent the change of pedestrians’ anticipation under the prominent impact of group behaviours, and predict pedestrians’ future states accurately. Moreover, the GIF contributes to explaining various anticipations of pedestrians’ behaviour in different social states. The proposed GIF can eventually innovate unmanned systems to work in a human-like manner, comply with social norms, and thereby promote a harmonious human–machine relationship.