Boosting Deep Reinforcement Learning with Semantic Knowledge for Robotic Manipulators

Deep Reinforcement Learning (DRL) is a powerful solution for complex sequential decision problems, especially in robotic control tasks. However, DRL’s effectiveness is often restrained by the extensive experience required for learning, leading to high computational and time costs. We present a novel integration of DRL and semantic knowledge through Knowledge Graphs Embeddings (KGEs), to enhance robotic control tasks by incorporating contextual information. Our approach leverages semantic knowledge from pre-built knowledge graphs to provide additional context to the learning agent, effectively reducing the sample complexity and improving learning efficiency. We propose a DRL agent architecture that concatenates KGEs with visual observations, enabling the agent to use those inputs and environmental knowledge. We validate our method through experiments with robotic manipulators in environments with fixed and randomized target attributes, demonstrating significant improvements in task performance, learning speed, and accuracy compared to baseline DRL agents without contextual information. This integration leads to a reduction in the learning time and, therefore, an improvement in sampling efficiency, up to 60%, and to an improvement in the agents’ accuracy of approximately 15 percentage points, without increasing the training time and computational complexity.