Ontology Supported Semantic Based Image Retrieval

Semantic-Based Image Retrieval (SBIR) is the process of searching for images in a database that have similar relationships between the objects detected in an image and the defined objects. The relationships in the subject-predicate-object structure are transformed into a structure that computers can make sense out of with Ontologies and are used to search for semantically similar images. In this study, a two-stage approach for developing a Semantic Based Image Retrieval system supported by Ontology is proposed. In the first stage, objects are detected with the Object Detection process from the image and a predicate describing the relationship between the two objects is determined with the developed Bi-directional Recurrent Neural Network (Bi-RNN) model. In the second stage, relations defined as are converted into Ontologies and used to search for semantically similar images. The Visual Genome dataset is used in the training of the developed Bi-RNN model and generation of the ontologies. In the performance measurement of the developed model, 91\% accuracy was obtained according to the Recall@100 (Top-5 accuracy) result. The proposed approach has the characteristics of a new method used in this field and gives more effective results compared to other similar methods that are used in Semantic Based Image Retrieval by using Ontologies.