Efficient Object-Level Semantic Mapping with RGB-D Cameras

Abstract To autonomously navigate in real-world environments, mobile robots require a dense map to guarantee safety, such as a 3D occupancy map. However, this map lacks semantic information for scene understanding. On the other hand, semantic objects can be introduced to the map with the help of deep neural networks, but they may suffer from critical run-time issues due to heavy processing components. In this paper, we present an efficient semantic mapping system to incrementally build a voxel-based map with individual objects. Firstly, a frame-wise object segmentation scheme is adopted to segment 3D objects from RGB-D images. Then, a new object association strategy with geometry and semantic descriptor is proposed to track and update object information, Finally, these objects are integrated into a CPU-based voxel mapping approach to incrementally build a global object-level volumetric map. Experiments on publicly available indoor datasets show that the proposed system achieves a good semantic mapping performance. Besides, our method outperforms other object-level mapping algorithms in terms of segmentation results and computational efficiency. Furthermore, the system is evaluated within a logistical robotic platform to demonstrate the use case in real-world applications.