A deep learning-based technique for firm annotation and domain adaptation in land cover classification using time-series aerial images

Abstract In this manuscript, a novel framework has been presented for firm annotation (classification) of a geographical area based on spatial as well as time-series analysis of multi-temporal very high resolution (VHR) satellite images. For this dual objective, an attention-based deep learning mechanism combined with the capabilities of convolutional-recurrent neural networks has been investigated for this purpose. The proposed annotation strategy is introduced as 'firm' since it allows the classifier to automatically ascertain the land-cover class by taking into consideration the geophysical changes on a landmass and thus outsmarting the conventional techniques relying on visual interpretation of a single image. The attention mechanism focuses on the important portions of the image scene while the convolutional long short-term memory neural networks exploit the temporal dependencies on the time-series image scenes. Moreover, an adaptive land cover classification scheme, considering the features extracted from the proposed classification approach has been explored for more robust time-series based firm annotation. To assess the performance of the proposed schemes, the experiments have been conducted on the two novels VHR multi-temporal land cover classification datasets. The investigated models have been shown to have the capacity to outperform the other state-of-the-art techniques under non-adaptive as well as adaptive scenarios using the multi-temporal images captured over disjoint geographical locations.