A blind area information perception and AR assembly guidance method based on RGBD data for dynamic environments and user study

Abstract In this research, a blind area information perception and guidance approach for dynamic context is proposed as a solution to the issue of difficult and time-consuming assembly in blind areas. The proposed approach involves the utilization of real-time RGBD data to perceive both blind area context and operator hand information. The resulting data is then used to visualize the blind area scene and provide assembly guidance through the application of augmented reality technology. Unlike conventional methods, the proposed solutions are based on dynamic RGBD data rather than static predefined CAD models, making it simpler to configure and adapt to more scenarios. A user study was designed and conducted to confirm the feasibility of the suggested approach. The results indicate that the suggested approach can decrease assembly time by 49.5%, greatly lower the percentage of assembly errors, reduce the mental load on the workers, and significantly enhance their operational experience.