Visual quality and safety monitoring system for human-robot cooperation

Abstract Efficient workspace awareness is critical for improved interaction in cooperative and collaborative robotics applications. In addition to safety and control aspects, quality-related tasks such as the monitoring of manual activities and the final quality assessment of the results are also required. In this context, a visual quality and safety monitoring system is developed and evaluated. The system integrates close-up observation of manual activities and posture monitoring. A compact single-camera stereo vision system and a time-of-flight depth camera are used to minimize the interference of the sensors with the operator and the workplace. Data processing is based on a deep learning to detect classes related to quality and safety aspects. The operation of the system is evaluated while monitoring a human-robot manual assembly task. The results show that the proposed system ensures a high level of safety, provides reliable visual feedback to the operator on errors in the assembly process, and inspects the finished assembly with a low critical error rate.