A novel method for image captioning using multimodal feature fusion employing mask recurrent neural networks and long short term memory

The image captioning is a technique that allows us to use computers to interpret the information of photographs and make written text. The use of deep learning to interpret image information and create descriptive text has become a widely researched issue since its establishment. Nevertheless, these strategies do not identify all samples that depict conceptual ideas. By reality, the vast majority of them seem to be irrelevant to the matching tasks. The degree of similarity is determined only by a few relevant semantics occurrences. These duplicate instances can also be thought of as noise, as they obstruct the matching process of a few meaningful instances and add to the model's computational effort. Furthermore, present approaches frequently require deliberate use of additional target recognition algorithms or costly human labelling in the extracting information is required.For image captioning, this research presents a multi-modal feature fusion based deep learning model. The coding layer uses Mask Recurrent Neural Networks (Faster R-CNN), the long short-term memory (LSTM)-attend has been used to decoding, and the descriptive text is constructed.In deep learning, the model parameters are optimized through method gradients optimization. In the decoding layer, dense attention mechanisms can assist in minimizing non-salient data interruption and preferentially input the appropriate comprised for the decryption stage. The model's capacity to comprehend images and generate text is validated by the experimental data in the domain of generic images. This paper is implemented using Python frameworks and also evaluated using the performance metrics such as PSNR, RMSE, SSIM, Accuracy, Recall, F1-score and Precision.