Research on Worker Assembly Procedure Inspection Based on Improved YOLO and Hand Trajectory

Visual inspection detects manual assembly actions to improve quality. Existing methods mainly analyze behaviors in single assembly scenarios using action recognition algorithms, but they cannot adapt to dynamic product switching and varying station environments in manual assembly lines. This paper proposes an adaptive assembly inspection method based on hand motion trajectories. By constructing an assembly inspection model based on the "Component-Tool-Product" triplet, the YOLOv9 model is employed to capture the spatiotemporal hand trajectory data stream. The assembly process is divided into a sequence of transitions between different assembly regions based on the hand's trajectory. The system analyzes the real-time hand trajectories using Dynamic Time Warping (DTW) and dwell time algorithms to obtain real-time assembly flow, allowing for the detection of missed and misassembled components. Additionally, the frame difference method is introduced to extract hand motion features, and an Attention Feature Fusion (AFF) module is used to integrate multi-scale features with motion texture information, enhancing the performance of the object detection model. Experimental results show that the proposed algorithm effectively reduces the false detection rate, with an average detection accuracy of 96% for missed and misassembled behaviors. The improved YOLOv9 model achieves an [email protected] of 93.94%.