Pose-NET: Spatial-Temporal Graph Learner in Human-Object Interaction Detection

Abstract In this paper, we adopt an original off-the-shelf spatial-temporal graph learner and incorporate it into the task of Human-Object Interaction (HOI) detection. We note that previous HOI models did attempt to account for pose data and some even used shallow graph nets on this input. Nevertheless, such works never utilized state of the art nets (with their advances in pose learning) proposed in skeleton-based activity recognition. This work intends to address this deficiency and evaluate the effect of having a dedicated pose-learner in existing HOI models. Most spatial-temporal nets where, of course, developed for learning temporal data as well as spatial. Nevertheless, in practice, we can run them in still images wherein the T-Conv layer will play no part in the scoring mechanism. Our work builds upon [11, 18] to build a quadruple cue learner for image datasets and improves upon previous performance by leveraging the strong pose learning capabilities of ST-GCN. The incorporated changes are tested on the V-COCO dataset.