Multi-view image-text emotion analysis network

Abstract A range of techniques to convey emotions through visuals and texts have emerged as a result of the rise of social media and the diversification of information delivery channels. The study of social media emotion based on texts and images has grown in popularity in recent years. Multimodal sentiment analysis approaches have advanced quickly since deep learning and some fusion algorithms first appeared. Existing multimodal models, however, have the issue of overlooking the lack of alignment between various modalities and the insufficient extraction of information for a single view of an image, which produces less-than-ideal results.In order to capture the correlation between objects and scenes in images and text, we introduces a cross-modal alignment module into the Multi-view Image Text Sentiment Analysis Network (MITN). Additionally, a stacked pooling module is used to combine multimodal data. On the MVSA-Single and MVSA-Multi datasets, experimental results demonstrate that the MITN suggested in this paper outperforms the baseline model.