Uncertainty based Regional-Decision Fusion of Multispectral and nDSM for Ultra-High Resolution UAV Imagery

The recent advancement in the pattern recognition technique has demonstrated the superiority in remote sensing technology, where Deep neural network uses the spatial feature representation such as convolution neural network (CNN), to provide better generalization capability. Regardless of any CNN structure, the prediction always involves uncertainty and imprecision while classifying the ultra-high resolution (UHR) image. Our aim is two-fold: firstly, increase the reliability feature by performing the Dual-scale fusion via a modified Markov random field known as DuCNN-MMRF. Secondly, an integration framework was introduced to combine the multispectral image classification produced by DuCNN-MMRF and normalized-Digital Surface Model (nDSM) information, using a novel approach known as constraint-based Dempster Shafer theory (C-DST). C-DST targeted DuCNN-MMRF’s uncertain information (ambiguous information) and rectified it with complementary information. The complementary information (i.e., nDSM) was processed using conventional machine learning (ML) techniques such as Multilayer perceptron (MLP), Support vector machine (SVM) and Random Forest (RF). The integration approach also uses the Shannon entropy function to exploit the uncertain information of model prediction (i.e. DuCNN-MMRF) at the regional level and subdivide into the positive and negative regions. The positive region is trusted by DuCNN-MMRF, and the negative region was combined with DuCNN-MMRF and MLP (and others ML) in a restricted manner. An ultra-high resolution (UHR) images was collected with an unmanned aerial vehicle (UAV) in the semi-urban region of IIT, Dhanbad, India, and labeled with building features. In addition, ISPRS Potsdam dataset with five land cover classes (Tree, building, grassland, impervious surface, and car) was used. The proposed framework was tested with several combinations i.e. MMRF-SVM, MMRF-RF, MMRF-MLP, along with max-voting fusion. The MMRF-MLP yielded highest accuracy with 85.24% (F measure ) and 97.79%(OA), and 76.12%(F measure ) and 91.09%(OA), for study area and Potsdam dataset, respectively.