Stretcher: A new way to improve the efficiency of exploration

The six-degree-of-freedom robotic arm has excellent spatial flexibility, allowing it to reach any point in the workspace from any orientation, making it perfect for executing manipulating tasks. However, in reinforcement learning situations, when the arm comes to a self-constraint state, the flexibility may have adverse effects on environment exploration, significantly complicating the reinforcement learning process. In this paper, inspired by human arm movements, we present a novel method for fast estimating the flexibility of multi-axis robotic arms. By integrating the metrics in RL algorithm, we made the exploration process faster and smoother. We compared the training speed and the effectiveness of training after incorporating the metrics at various stages of the standard RL algorithm like observation and rewrd. After analysing the spatial flexibility, the arm is found to show improved exploration performance, as well as faster training speed and less temporal consumption during the movement transitions.