Enhancing Salient Product Detection: A HybridApproach Incorporating Salient Object Mask and Visual Explanations Heatmap

Abstract In online marketplaces, managing product images can be challenging due to the large volume of images and products available. To capture customers’ attention and promote sales, it is essential to effectively highlight and position the selling product in the ad image. In this paper, we propose a hybrid method for detecting salient product regions in images, which combines salient object detection with visual explanation techniques. Our approach leverages the strengths of two state-of-the-art models: U2-Net for generating high-quality salient object masks, and Ablation-CAM for generating visual explanation heatmaps that highlight the most discriminative regions for classification. According to the experimental results, the hybrid model reduces the number of images that require manual evaluation for determining the suitability between product categories and ad images from 5500 to 250. Furthermore, we demonstrate the interpretability of our method by visualizing the salient regions and explanation heatmaps, which can provide insights into the model’s decision-making process and help users understand why certain regions are important for classification. Overall, our approach offers an effective solution for automating the process of identifying and highlighting the most relevant product regions in ad images, thereby saving time and effort for online marketplace sellers.