Lidar-Camera Fusion For Road Detection Using Recurrent Conditional Random Field Model

Reliable road detection is an essential task in autonomous driving systems. Two categories of sensors are commonly used: camera and LiDAR, each of which can provide corresponding supplemental. Nevertheless, existing sensor fusion methods do not fully utilize multimodal data. Most of them are dominated by image or taking point cloud as a supplement rather than making the best of them, the correlation between modalities is ignored. This paper proposes Recurrent Conditional Random Field Model (R-CRF) for road detection by fusing image and point cloud segmentation. The R-CRF integrates results (information) from modalities in a probabilistic way. Each modality is independently processed with its own semantic segmentation network. The probability scores obtained are considered as unary term for individual pixel nodes in random field while RGB image and the densified LiDAR-images are used as pairwise terms. The energy function is then iteratively optimized by mean-field variational inference so that we can refine the labeling results by exploiting fully-connected graphs of the RGB image and LiDAR-images. Extensive experiments are conducted on KITTI-Road dataset, and these demonstrate the method achieves competitive performance.