Answer Me: Pictorial Interrogation on Categorized Queries

The recent years have witnessed massive development in the fields of computer vision, object recognition, and natural language processing (NLP). Artificial Intelligence (AI) applications use NLP to provide the computer with facilities, such as question-answering models. An extension of this method is to integrate NLP with computer vision to perform the Visual Question Answering (VQA) mission, which is to construct systems that can respond to image questions in natural language. In this paper, the application of Convolutional Neural Network (CNN) for the image Question Answering (QA) task is proposed. The proposed CNN provides an end-to-end framework with convolutional architectures for learning not only the image and question representations, but also their inter-modal interactions to produce the answer. More specifically, this model consists of three CNNs: image CNN to encode the image content, sentence CNN to compose the words of the question, and multimodal convolution layer to learn their joint representation for the classification in the space of candidate answer words.