Deep Learning-based Pose Estimation in Providing Feedback for Physical Movement: a Review

Pose estimation has various applications in analyzing human movement and behavior, including providing feedback to users about their movements so they could adjust and improve their movement skills. To investigate the current research status and possible gaps, we searched Scopus and Web of Science for articles that (1) human `body' pose estimation is used and (2) user movement is assessed and communicated. We used either a bottom-up or top-down approach to analyze 20 articles for methods used to estimate human pose, assess movement, provide feedback to users, as well as methods to evaluate them. Our review found that pose estimation systems typically used CNNs while movement assessment methods varied from mathematical formulas or models, rule-based approaches, to machine learning. Feedback was primarily presented visually in verbal forms and nonverbal forms. The experiments to evaluate each part ranged from the use of public datasets to human participants. We found that while there was an improvement, the majority of pose estimation challenges remain. The effectiveness and factors for choosing movement assessment methods for a new context are still unclear. In the end, we suggest that studies about feedback prioritization and erroneous feedback are needed.