Incremental Learning Approach for Semantic Segmentation of Skin Histology Images

Abstract This study presents an incremental learning framework to enhance the generalization and robustness of transformer-based deep learning models for segmenting skin cancer and related tissue structures. While deep learning models often perform well on data distributions similar to their training sets, their accuracy typically degrades when exposed to novel scenarios—limiting their clinical utility in skin cancer diagnosis. To address this, we propose a biologically inspired incremental learning strategy tailored for skin cancer classification and segmentation, allowing the model to incorporate new data progressively while reducing catastrophic forgetting. Our approach integrates multiple loss functions to preserve existing knowledge while adapting to additional magnification levels. Experimental results on the in-distribution test set demonstrate consistent performance improvements: achieving 89.05% accuracy with 10× magnification, 92.68% with 10× and 5× combined, and 95.53% when incorporating 10×, 5×, and 2× magnifications. These findings highlight the potential of our method to improve the adaptability and reliability of deep learning systems for empirical generalization in skin cancer classification tasks. Please note Abbreviations should be introduced at the first mention in the main text – no abbreviations lists. Suggested structure of main text (not enforced) is provided below. Competing Interest Statement The authors have declared no competing interest. Funding Statement This study did not receive any funding. Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: This study used a skin histology dataset collected by Queensland University publicly available at https://espace.library.uq.edu.au/view/UQ:8be4bd0. Access was granted through the dataset providers after application. Ethical approval and informed consent were obtained by the original dataset providers. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Footnotes email:(sfatima.ce22ceme{at}student.nust.edu.pk), (anum.abdulsalam{at}ceme.nust.edu.pk, usman.akram{at}ceme.nust.edu.pk) email:(ibrahim.abdelhameed{at}nmbu.no) email: (sfatima7{at}lakeheadu.ca, sbinahm{at}lakeheadu.ca) Data Availability The dataset analyzed in this study was provided by Queensland University and is available upon request from the data providers. All additional results generated during this study are contained within the manuscript.