Fundus Image Analysis of Retinitis Pigmentosa Using Artificial Intelligence

Fundus Image Analysis of Retinitis Pigmentosa Using Artificial Intelligence | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Fundus Image Analysis of Retinitis Pigmentosa Using Artificial Intelligence Saki Ubukata, Kanato Masayoshi, Yusaku Katada, Lizhu Yang, Nobuhiro Ozawa, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4851616/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Retinitis pigmentosa (RP) is one group of inherited retinal diseases that are caused by genetic defects that lead to progressive photoreceptor loss and eventual blindness. Early diagnosis will helpful for a effective management of the disease, however, many patients remain unaware of eraly symptoms. Meanwhile, fundus images are widely taken for medical checkups, however, are underused in detecting RP. This study explores the potential of deep learning to identify RP from color fundus images. The dataset contained 200 color fundus images of Japanese RP patients and 121 color fundus images from non-RP subjects from Keio University Hospital. Using transfer learning, pretrained convolutional neural network models -VGG16, Resnet50, and InceptionV3- were finetuned to detect RP. As a result, Inception V3 achieved the best accuracy of 96.97%, which matches the average diagnostic accuracy of ophthalmologists. Using Gradient-weighted Class Activation Mapping (Grad-CAM), we identified peripheral pigmentation in the fundus images as a critical feature for diagnosis, aligning with the known progression patterns of RP. This confirms the robustness and validity of our model, highlighting the utility of deep learning in assisting ophthalmologists with RP screening. Health sciences/Diseases/Eye diseases Health sciences/Diseases/Eye diseases/Retinal diseases retinitis pigmentosa deep learning fundus images artificial intelligence Figures Figure 1 Figure 2 Figure 3 Figure 4 1. Introduction Retinitis pigmentosa (RP) is an intractable and serious eye disease that affects approximately 2 million people worldwide [ 1 ]. RP is caused by degeneration of photoreceptors in the retina, resulting in the appearance of abnormal findings on fundus images, mainly pigmentation, in the periphery. As the pigmentation progresses, visual field testing and fundus imaging are performed to monitor the progress of the disease. Although there is no effective treatment for RP at this moment, research and development are actively underway [ 2 , 3 ]. Thus, it is important to diagnose and predict RP to smoothly guide potential future treatment. Early diagnosis also has the potential to reduce psychological burden by allowing more time to prepare for visual impairment in the future. However, detecting RP in its early stages poses a challenge due to its limited subjective symptoms. While color fundus images offer a cost-effective and noninvasive means of identifying RP, the scarcity of ophthalmologists makes widespread screening impractical in clinical settings. Consequently, leveraging AI for RP screening would be highly beneficial. Fundus images contain vast amounts of data, making it challenging to identify subtle abnormalities during routine examinations. Therefore, artificial intelligence can assist in detecting such anomalies within fundus images promptly and accurately, aiding in early intervention. Given the above, there would be a tremendous benefit if AI could be utilized to assist in the early diagnosis of RP. With the rapid progress of deep learning study, research on the application of artificial intelligence to the diagnosis of various diseases, including ophthalmology diseases, has been active in recent years [ 4 , 5 ]. Previous studies have investigated the application of deep learning techniques for the classification of RP using fundus images. Guo C et al. Eexplored various eye diseases, including RP, using deep transfer methods [ 6 ]. However, it is noteworthy that this study did not incorporate cross-validation techniques. Similarly, another study by Chen TC et al. reported sensitivity and specificity of 91.2% and 91.71%, respectively, for the early detection of RP using AI. While these findings demonstrate certain levels of accuracy, they also suggest the potential for further improvement in diagnostic performance [ 7 ]. In contrast, our study aims to build upon these prior works by not only employing deep learning algorithms for RP classification but also conducting thorough cross-validation procedures to ensure the robustness and generalizability of our model. Through this approach, we aim to achieve enhanced sensitivity and specificity in the early detection of RP. Furthermore, we also executed gradient-weighted class activation mapping (Grad-CAM), [ 8 ]which shows important pixels for prediction to confirm whether our program will be able to apply to unknown fundus images. 2. Material and Methods 2.1 Dataset In this study, a program for predicting RP using color fundus images is designed as shown in Figure 1 . This study was approved by the Ethics Review Committee of Keio University School of Medicine (Approval No. 20210809) and conducted in accordance with the Declaration of Helsinki. Written informed consent was waived due to retrospective nature of the study. The patient's personal information was removed from the images, prior to the AI analysis. We collected RP and non-RP images taken at Keio University Hospital during January 2012 through May 2021. The images were then reviewed by ophthalmologists and poor-quality images such as significant blur and wrong angles of views were removed from the dataset. After that, the images were cropped to the smallest long rectangle that would accommodate the oval shape of the fundus and resized to a (256, 256) size. Since the dataset was small, the data was augmented by randomly flipping and changing contrast. Augmentation was applied only for the training dataset. Table 1 Patient demographic characteristics. Patient demographic data were presented in Table 1. RP Non-RP Total 107 94 Male 56 47 Female 51 47 Age mean [range] 52.8 [17-87] 54.4 [23-81] 2.2 Model In this study, deep learning transfer learning techniques with three convolutional neural networks, the VGG16, Inception V3, and Resnet 50 were used. VGG16 is composed of 13 convolutional layers, 5 max-pooling layers, and 3 fully connected layers [9]. InceptionV3 is 42 layers deep and it is using several approaches to reduce the total number of parameters [10]. Resnet 50 has 48 convolutional layers along with 1 maxpool layer and 1 average pool layer [11]. Those are pre-trained on more than a million images from the ImageNet dataset [12]. Adam optimizer was used for the optimization for all models. The learning rate was 0.000001, the number of training epochs was 200, and loss function was cross entropy. The used fundus images comprise a total of 201 individuals, with 107 individuals diagnosed with RP and 94 individuals classified as non-RP. 2.3 Evaluation In this study, we adopted a two-stage evaluation process, where we first assessed the performance of our model using cross-validation (CV) and subsequently conducted a final evaluation using a separate test dataset. This approach ensures robustness and reliability in our assessment, as it allows us to validate the generalizability of our model's performance beyond the training data. We conducted a final evaluation on an isolated test dataset to provide a comprehensive and trustworthy assessment of our AI models.We utilized evaluation metrics, including the receiver operating characteristic (ROC) curve, and its area under the curve (AUC), to assess the performance of our model. We used PyTorch to build a model and Matplotlib for visualization. The training and inference were executed with NVIDIA V100 Tensor Core GPU. 2.4 Visualization We visualized which parts of the image had the greatest impact on the prediction using Grad-CAM, which is a method for providing an explanation for a given input and its prediction by a CNN-based image recognition model. Ophthalmologists conducted a validity assessment and error analysis of the regions of interest identified based on this map. 2.5 For further examination of the performance and usefulness of the model, a comparison was made with the performance of ophthalmologists and medical students in classification. The model, four ophthalmologists, and one medical student performed classification using the same 66 images as test data. Figure 1. Program Flow Chart. The dataset containing 321 images was separated into the training and test dataset. We evaluated the models by 5-fold cross-validation on the training dataset and hold-out method using the test dataset. 3. Results 3.1 Image collection We collected 411 fundus images and excluded 90 images due to poor quality. As a result, our dataset contained 200 RP fundus images from 107 patients and 121 non-RP fundus images from 94 patients. The collected 321 images consisting of RP and non-RP fundus images were separated into a training set (159 RP and 96 non-RP) and a test set (41 RP and 25 non-RP). The patient demographic characteristics are shown in Table 1 . 3.2 5-fold cross validation In comparing the three CNNs used in training the model, 5-fold cross validation showed that Inception V3 attained the best performance in accuracy ( Table 1 ). For VGG16 the accuracy was 0.749 ± 0.110, for Resnet50 0.883 ± 0.120, and for InceptionV3 0.897 ± 0.078. After training, we conducted testing on the test dataset. Among these, the results of InceptionV3 yielded the best performance, as demonstrated below. As Tabel 2 shows, the model made wrong precisions only for 2 non-RP images. For the rest of the images in the dataset (41 RP images and 22 non-RP images), the model made correct prediction ( Table 3 ). Table 2 Performance metrics of deep learning models The accuracy and standard deviation (SD) obtained from cross-validation, presented as a percentage. Cross-validation involves dividing the dataset into multiple smaller groups, using each group as a test set to evaluate the model's generalizability and reliability. The AUROC (Area Under the Receiver Operating Characteristic Curve) value obtained on the test dataset, presented as a percentage. The AUROC value measures how well the model can distinguish between different classes, with higher values indicating better performance. Model Cross Validation Accuracy (± SD), % Test Dataset AUROC, % VGG16 0.749 ± 0.110 95.21 Resnet50 0.883 ± 0.120 97.85 Inception V3 0.897 ± 0.078 99.32 Table 3 Confusion matrix with Inception V3. This table shows the performance of the Inception V3 model in classifying data into two categories, RP and Non-RP. Predicted Label RP Non-RP True Label RP 41 0 No RP 2 23 Figure2 Receiver operating characteristic (ROC) curve of Inception V3 model with plots of the average of ophthalmologists and a student. ROC curve of Inception V3 model, with performance comparisons to average ophthalmologists and a student, depicted by orange and green triangles respectively. The horizontal axis shows False Positive Rate (FPR) and the vertical axis shows True Positive Rate (TPR). Figure 2 shows the performance of the Inception V3 model along with the ophthalmologists and medical student accuracy. Inception V3 scored best in AUROC compared with two other models shown in Table 2 . 3.3 Visualization We conducted a Grad-CAM to visualize which pixels the model sees as important in the prediction process for the model based on Inception V3 to confirm the validity of the model to show that it did not rely on features unrelated to RP. Figure 3 Grad-CAM analysis of misdiagnosed samples . Representative original fundus color images and corresponding Grad-CAM heatmaps are shown for each diagnostic category. The Grad-CAM heatmap show the regions that contributed significantly to the model's decision: red and yellow indicate high contribution, blue indicates low contribution. Our analysis revealed that the heatmap predominantly focus on the peripheral regions. However, instances of false positives were observed to be centered around the macular region, indicating potential misfocus. The model correctly identified all RP images. Figure 4 shows two RP images that was misdiagnosed as non-RP by the ophthalmologist but was correctly predicted to be RP by this program. As shown in this example, the program was able to correctly diagnose RP images that would have been missed by an ophthalmologist in a non-RP screening. Figure 4. RP Images that only AI could correctly diagnose Representative original fundus color images and corresponding Grad-CAM heat maps are shown for each diagnostic category. 3.4 Performance comparison with Ophthalmologists and a Medical student The model and ophthalmologists demonstrated comparable performance in terms of accuracy, recall, specificity, and precision. Additionally, the model outperformed medical students in RP detection. The results are shown in Table 4. The performance of the model was s comparable to that of ophthalmologists and higher than that of a medical student. Table 4 Performance comparison of our AI model, ophthalmologist, and medical student. The table displays accuracy, recall, specificity, and precision for four ophthalmologists (labeled 1-4), their mean values, a medical student, and a machine learning model. Accuracy, % Recall, % Specificity, % Precision, % Ophthalmologist 1 95.45 92.68 100 100 Ophthalmologist 2 98.48 97.56 100 100 Ophthalmologist 3 96.97 95.12 100 100 Ophthalmologist 4 96.97 95.12 100 100 Ophthalmologists Mean 96.97 95.12 100 100 Medical student 81.82 73.17 96 96.77 Model 96.97 95.35 100 100 4. Discussion 4.1 Performance RP is a retinal degenerative disease that can lead to loss of sight. Early detection is important in terms of reducing the psychological burden of possible future disabilities even if there is no effective treatment for now. The model using AI can be useful to help ophthalmologists find RP patients. We experimented with VGG16, Resnet50, and Inception V3 to construct the model. As a result, Inception V3 reached the highest accuracy. Since Resnet50 and Inception V3 are improved networks of VGG16, they performed better than VGG16. For an image classification task like this, features can exist anywhere in the image at various sizes. That is why choosing a kernel size is difficult. Inception V3 solves this problem by setting filters of various size at the same level at the same time instead of choosing one. This is probably the reason why Inception V3 could achieve the best performance. In comparison to the prior study conducted by Chen et al., which reported sensitivity and specificity values of 91.2% and 91.71%, respectively, we achieved higher sensitivity and specificity rates in our classification model. This improvement underscores the effectiveness of our approach in accurately identifying individuals with RP, thereby highlighting the potential clinical utility. Our Grad-Cam analysis, as shown in Fig. 3 , revealed that the heatmap predominantly highlights the peripheral regions. This inclination can be rationalized by the typical progression of RP from the periphery inward, leading to more prominent degenerative changes in peripheral areas. Consequently, the focus on these regions appears to be a logical outcome. In the two false positive cases, the program primarily concentrated on the central area of the image. The probable cause for this misfocus might arise from the misinterpretation of the macular Tigroid pattern as retinal degenerative changes. Additionally, these errors could stem from insufficient data. These findings underscore the necessity for further exploration into the factors contributing to misfocusing, particularly in distinguishing macular patterns from degenerative changes, to refine the accuracy of the classification model. In summary of the visualization, it can be inferred that the model predominantly directs attention to pigmentation in the periphery and the macula for prediction, mirroring the approach of ophthalmologists. This suggests the potential for accurate classification of unknown fundus images. Going back to the performance of the model was as high as that of the ophthalmologists’ diagnoses. The reason for the high performance despite the relatively small amount of data is thought to be that the abnormal findings were relatively easy to detect. 4.2 Limitations and future research Our work has several limitations. First, the number of images used in the study was limited, and further accumulation of training data is needed to achieve a practical level. Second, this model only detects the presence or absence of RP and cannot determine the causative gene, stage of the disease, or prognosis. Third, All images used in this study were only taken at Keio University Hospital. To increase the versatility of this model for clinical application, it is necessary to collect data not from a single institution but from multiple institutions. 5. Conclusion This study is the first to report the development of an AI to diagnose RP from fundus images of Japanese patients. It achieved results comparable to ophthalmologists' diagnoses. Additionally, by utilizing Grad-CAM, we have confirmed that the model prioritizes the peripheral areas, aligning with the known progression pattern of RP. We also discovered that the model may mistakenly identify macular patterns as indicative of RP The results of this study suggest the usefulness of using deep learning for the automatic determination of RP or tools to reduce the burden on ophthalmologists. Abbreviations AI; Artificial Intelligence, AUC; Area Under Cover, FPR; False Positive Rate, GradCAM; gradient-weighted class activation mapping, ROC; Receiver Operating Characteristic, RP; retinitis pigmentosa, TPR; True Positive Rat Declarations Acknowledgments The authors thank the members of the Laboratory of Photobiology, Keio University School of Medicine for their technical and administrative support. Especially, we would like to express our gratitude to those who provide fundus images and who were involved in clinical practice at photographed the data. Author contributions Saki Ubukata : Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing - Original Draft, Visualization Kanato Masayoshi: Conceptualization, Methodology, Software, Formal analysis, Investigation, Visualization Yusaku Katada : Conceptualization, Methodology, Resources, Data Curation, Writing - Review & Editing, Funding acquisition Lizhu Yang: Methodology, Resources, Data Curation, Writing - Review & Editing Nobuhiro Ozawa : Resources, Data Curation, Writing - Review & Editing, Funding acquisition Mari Ibuki : Resources, Data Curation, Writing - Review & Editing, Funding acquisition Kazuno Negishi : Writing - Review & Editing, Supervision Toshihide Kurihara : Writing - Review & Editing, Supervision Data availability The dataset is not publicly available due to its containing information that could compromise the privacy of research participants but are available from the corresponding author upon reasonable request. Competing interests The authors declare no competing interests. References Curr Genomics. 2011;12(4):260–266. Prado DA, Acosta-Acero M, Maldonado RS. Gene therapy beyond luxturna: a new horizon of the treatment for inherited retinal disease. Curr Opin Ophthalmol. 2020;31(3):147–154. Miraldi Utz V, Coussa RG, Antaki F, Traboulsi EI. Gene therapy for RPE65-related retinal disease. Ophthalmic Genet. 2018;39(6):671–677. Ting DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103(2):167–175. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–2410. Chen G, Yu M, Li J. Prediction of different eye diseases based on fundus photography via deep transfer learning. J Clin Med. 2021;10(230):5481. Chen TC, Lim WS, Wang VY, Ko ML, Chiu SI, Huang YS, Lai F, Yang CM, Hu FR, Jang JSR, Yang CH. Artificial intelligence–assisted early detection of retinitis pigmentosa—the most common inherited retinal degeneration. J Digit Imaging. 2021;34(4): https://doi.org/10.1007/s10278-021-00479-6 . Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 618–626. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Preprint at https://arxiv.org/abs/1409.1556 . Accessed 19 Jan 2021. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015:2818–2826. Targ S, Almeida D, Lyman K. Resnet in resnet: generalizing residual architectures. Preprint at https://arxiv.org/abs/1603.08029 . Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR09), Miami, FL, USA, 20–25 June 2009. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4851616","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":345458117,"identity":"6ab7e2c9-d2bf-48e7-b413-c9150d0d1a98","order_by":0,"name":"Saki Ubukata","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Saki","middleName":"","lastName":"Ubukata","suffix":""},{"id":345458118,"identity":"d0922364-b14a-43dd-a69e-b4303cbcdfcd","order_by":1,"name":"Kanato Masayoshi","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Kanato","middleName":"","lastName":"Masayoshi","suffix":""},{"id":345458119,"identity":"803c05b5-5fce-491d-91da-e73865713efc","order_by":2,"name":"Yusaku Katada","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Yusaku","middleName":"","lastName":"Katada","suffix":""},{"id":345458120,"identity":"dfc60a88-4763-4af8-84aa-f1a48b536951","order_by":3,"name":"Lizhu Yang","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Lizhu","middleName":"","lastName":"Yang","suffix":""},{"id":345458121,"identity":"6b81f5cb-f980-4957-b855-6a7384b77880","order_by":4,"name":"Nobuhiro Ozawa","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Nobuhiro","middleName":"","lastName":"Ozawa","suffix":""},{"id":345458122,"identity":"42b28918-a0fb-49a6-8dad-d833db622f26","order_by":5,"name":"Mari Ibuki","email":"","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Mari","middleName":"","lastName":"Ibuki","suffix":""},{"id":345458124,"identity":"57d18902-7861-4313-9395-a9d874f342d9","order_by":6,"name":"Kazuno Negishi","email":"","orcid":"","institution":"Department of Ophthalmology, Keio University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Kazuno","middleName":"","lastName":"Negishi","suffix":""},{"id":345458125,"identity":"b460dfde-9cdb-4761-9666-e5129b857451","order_by":7,"name":"Toshihide Kurihara","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA20lEQVRIiWNgGAWjYBACCSA+DGLwMzAfYGwA8w8QqUWygS2BeC3MIIbBAR4DoBYigOSM3IeHC2ruyRufP5bAOKPCQo6B8Sx+a6Ql0g0OzzhWbLjtwOEDjBvOSBgzMJxLwKtFTiKN4TAP0BvbDrYlMD5sk0hsYDhjQISWfwn2m5uBfnn4jwgt0iAtvG0JiRvYgFo2NhChRbLnGVBLX0LyjDNsCQdnHJMwZiPkF4njacyfeb4l2Pb3Hz74sKemTo5fgkCIoQCwUjaJM8TrgAL+HpK1jIJRMApGwfAGAAkVR/qcREc+AAAAAElFTkSuQmCC","orcid":"","institution":"Laboratory of Photobiology, Keio University School of Medicine","correspondingAuthor":true,"prefix":"","firstName":"Toshihide","middleName":"","lastName":"Kurihara","suffix":""}],"badges":[],"createdAt":"2024-08-03 06:08:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4851616/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4851616/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":64164588,"identity":"e5d5d49f-6722-4c95-9650-e3c8e3bf0821","added_by":"auto","created_at":"2024-09-09 08:52:08","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":115831,"visible":true,"origin":"","legend":"\u003cp\u003eProgram Flow Chart. The dataset containing 321 images was separated into the training and test dataset. We evaluated the models by 5-fold cross-validation on the training dataset and hold-out method using the test dataset.\u003c/p\u003e","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4851616/v1/b18ab11ae64e8b8d61127ada.jpeg"},{"id":64164591,"identity":"60f663ef-2b15-45e4-b398-61c0afe69f74","added_by":"auto","created_at":"2024-09-09 08:52:08","extension":"jpeg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":85318,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eReceiver operating characteristic (ROC) curve of Inception V3 model with plots of the average of ophthalmologists and a student.\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4851616/v1/7abbe58b6f9b61c6881ef760.jpeg"},{"id":64164946,"identity":"6baf60b9-c6e5-4633-acea-aa147d335a3c","added_by":"auto","created_at":"2024-09-09 09:00:08","extension":"jpeg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":553208,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eGrad-CAM analysis of misdiagnosed samples\u003c/strong\u003e. Representative original fundus color images and corresponding Grad-CAM heatmaps are shown for each diagnostic category. The Grad-CAM heatmap show the regions that contributed significantly to the model's decision: red and yellow indicate high contribution, blue indicates low contribution.\u003c/p\u003e","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4851616/v1/8afa7493e45a2920e460ad1e.jpeg"},{"id":64164590,"identity":"417acc24-f97b-43bf-b752-c60d501de2eb","added_by":"auto","created_at":"2024-09-09 08:52:08","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":374791,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRP Images that only AI could correctly diagnose\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"floatimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4851616/v1/76274116b8fd985c4581e239.jpeg"},{"id":90633252,"identity":"07ab93f8-b162-4cdb-b1d4-3a7c7c8be619","added_by":"auto","created_at":"2025-09-05 03:31:42","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2010498,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4851616/v1/3f1cd53e-23d7-4533-8243-5980073861f4.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Fundus Image Analysis of Retinitis Pigmentosa Using Artificial Intelligence","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eRetinitis pigmentosa (RP) is an intractable and serious eye disease that affects approximately 2\u0026nbsp;million people worldwide [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. RP is caused by degeneration of photoreceptors in the retina, resulting in the appearance of abnormal findings on fundus images, mainly pigmentation, in the periphery. As the pigmentation progresses, visual field testing and fundus imaging are performed to monitor the progress of the disease. Although there is no effective treatment for RP at this moment, research and development are actively underway [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Thus, it is important to diagnose and predict RP to smoothly guide potential future treatment. Early diagnosis also has the potential to reduce psychological burden by allowing more time to prepare for visual impairment in the future. However, detecting RP in its early stages poses a challenge due to its limited subjective symptoms. While color fundus images offer a cost-effective and noninvasive means of identifying RP, the scarcity of ophthalmologists makes widespread screening impractical in clinical settings. Consequently, leveraging AI for RP screening would be highly beneficial. Fundus images contain vast amounts of data, making it challenging to identify subtle abnormalities during routine examinations. Therefore, artificial intelligence can assist in detecting such anomalies within fundus images promptly and accurately, aiding in early intervention. Given the above, there would be a tremendous benefit if AI could be utilized to assist in the early diagnosis of RP.\u003c/p\u003e \u003cp\u003eWith the rapid progress of deep learning study, research on the application of artificial intelligence to the diagnosis of various diseases, including ophthalmology diseases, has been active in recent years [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Previous studies have investigated the application of deep learning techniques for the classification of RP using fundus images. Guo C et al. Eexplored various eye diseases, including RP, using deep transfer methods [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. However, it is noteworthy that this study did not incorporate cross-validation techniques. Similarly, another study by Chen TC et al. reported sensitivity and specificity of 91.2% and 91.71%, respectively, for the early detection of RP using AI. While these findings demonstrate certain levels of accuracy, they also suggest the potential for further improvement in diagnostic performance [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. In contrast, our study aims to build upon these prior works by not only employing deep learning algorithms for RP classification but also conducting thorough cross-validation procedures to ensure the robustness and generalizability of our model. Through this approach, we aim to achieve enhanced sensitivity and specificity in the early detection of RP. Furthermore, we also executed gradient-weighted class activation mapping (Grad-CAM), [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]which shows important pixels for prediction to confirm whether our program will be able to apply to unknown fundus images.\u003c/p\u003e"},{"header":"2. Material and Methods","content":"\u003ch2\u003e2.1 Dataset\u003c/h2\u003e\n\u003cp\u003eIn this study, a program for predicting RP using color fundus images is designed as shown in \u003cstrong\u003eFigure 1\u003c/strong\u003e. This study was approved by the Ethics Review Committee of Keio University School of Medicine (Approval No. 20210809) and conducted in accordance with the Declaration of Helsinki. Written informed consent was waived due to retrospective nature of the study. The patient\u0026apos;s personal information was removed from the images, prior to the AI analysis.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe collected RP and non-RP images taken at Keio University Hospital during January 2012\u0026nbsp;through May 2021. The images were then reviewed by ophthalmologists and poor-quality images such as significant blur and wrong angles of views were removed from the dataset. After that, the images were cropped to the smallest long rectangle that would accommodate the oval shape of the fundus and resized to a (256, 256) size. Since the dataset was small, the data was augmented by randomly flipping and changing contrast. Augmentation was applied only for the training dataset.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1 Patient demographic characteristics.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003ePatient demographic data were presented in Table 1.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"435\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"4.137931034482759%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"27.816091954022987%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"32.18390804597701%\"\u003e\n \u003cp\u003eRP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.86206896551724%\"\u003e\n \u003cp\u003eNon-RP\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"31.95402298850575%\" colspan=\"2\"\u003e\n \u003cp\u003eTotal\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"32.18390804597701%\"\u003e\n \u003cp\u003e107\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.86206896551724%\"\u003e\n \u003cp\u003e94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"4.137931034482759%\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd width=\"27.816091954022987%\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"32.18390804597701%\"\u003e\n \u003cp\u003e56\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.86206896551724%\"\u003e\n \u003cp\u003e47\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"4.137931034482759%\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd width=\"27.816091954022987%\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"32.18390804597701%\"\u003e\n \u003cp\u003e51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.86206896551724%\"\u003e\n \u003cp\u003e47\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"31.95402298850575%\" colspan=\"2\"\u003e\n \u003cp\u003eAge mean [range]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"32.18390804597701%\"\u003e\n \u003cp\u003e52.8 [17-87]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.86206896551724%\"\u003e\n \u003cp\u003e54.4 [23-81]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2\u003e2.2 Model\u003c/h2\u003e\n\u003cp\u003eIn this study, deep learning transfer learning techniques with three convolutional neural networks, the VGG16, Inception V3, and Resnet 50 were used. VGG16 is composed of 13 convolutional layers, 5 max-pooling layers, and 3 fully connected layers [9]. InceptionV3 is 42 layers deep and it is using several approaches to reduce the total number of parameters [10]. Resnet 50 has 48 convolutional layers along with 1 maxpool layer and 1 average pool layer [11]. Those are pre-trained on more than a million images from the ImageNet dataset [12]. Adam optimizer was used for the optimization for all models. The learning rate was 0.000001, the number of training epochs was 200, and loss function was cross entropy.\u0026nbsp;The used fundus images comprise a total of 201 individuals, with 107 individuals diagnosed with RP and 94 individuals classified as non-RP.\u003c/p\u003e\n\u003ch2\u003e2.3 Evaluation\u003c/h2\u003e\n\u003cp\u003eIn this study, we adopted a two-stage evaluation process, where we first assessed the performance of our model using cross-validation (CV) and subsequently conducted a final evaluation using a separate test dataset. This approach ensures robustness and reliability in our assessment, as it allows us to validate the generalizability of our model\u0026apos;s performance beyond the training data. We conducted a final evaluation on an isolated test dataset to provide a comprehensive and trustworthy assessment of our AI models.We utilized evaluation metrics, including the receiver operating characteristic (ROC) curve, and its area under the curve (AUC), to assess the performance of our model.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe used PyTorch to build a model and Matplotlib for visualization. The training and inference were executed with NVIDIA V100 Tensor Core GPU.\u003c/p\u003e\n\u003ch2\u003e2.4 Visualization\u003c/h2\u003e\n\u003cp\u003eWe visualized which parts of the image had the greatest impact on the prediction using Grad-CAM, which is a method for providing an explanation for a given input and its prediction by a CNN-based image recognition model. Ophthalmologists conducted a validity assessment and error analysis of the regions of interest identified based on this map. 2.5\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFor further examination of the performance and usefulness of the model, a comparison was made with the performance of ophthalmologists and medical students in classification. The model, four ophthalmologists, and one medical student performed classification using the same 66 images as test data.\u003c/p\u003e\n\u003cp\u003eFigure 1. Program Flow Chart. The dataset containing 321 images was separated into the training and test dataset. We evaluated the models by 5-fold cross-validation on the training dataset and hold-out method using the test dataset.\u003c/p\u003e"},{"header":"3. Results","content":"\u003ch2\u003e3.1 Image collection\u003c/h2\u003e\n\u003cp\u003eWe collected 411 fundus images and excluded 90 images due to poor quality. As a result, our dataset contained 200 RP fundus images from 107 patients and 121 non-RP fundus images from 94 patients. The collected 321 images consisting of RP and non-RP fundus images were separated into a training set (159 RP and 96 non-RP) and a test set (41 RP and 25 non-RP). The patient demographic characteristics are shown in \u003cstrong\u003eTable 1\u003c/strong\u003e.\u003c/p\u003e\n\u003ch2\u003e3.2 5-fold cross validation\u003c/h2\u003e\n\u003cp\u003eIn comparing the three CNNs used in training the model, 5-fold cross validation showed that Inception V3 attained the best performance in accuracy (\u003cstrong\u003eTable 1\u003c/strong\u003e). For VGG16 the accuracy was 0.749 \u0026plusmn; 0.110, for Resnet50 0.883 \u0026plusmn; 0.120, and for InceptionV3 0.897 \u0026plusmn; 0.078.\u003c/p\u003e\n\u003cp\u003eAfter training,\u0026nbsp;we conducted testing on the test dataset. Among these, the results of InceptionV3 yielded the best performance, as demonstrated below.\u0026nbsp;As \u003cstrong\u003eTabel 2\u003c/strong\u003e shows, the model made wrong precisions only for 2 non-RP images. For the rest of the images in the dataset (41 RP images and 22 non-RP images), the model made correct prediction (\u003cstrong\u003eTable 3\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;Table 2 Performance metrics of deep learning models\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe accuracy and standard deviation (SD) obtained from cross-validation, presented as a percentage. Cross-validation involves dividing the dataset into multiple smaller groups, using each group as a test set to evaluate the model\u0026apos;s generalizability and reliability. The AUROC (Area Under the Receiver Operating Characteristic Curve) value obtained on the test dataset, presented as a percentage. The AUROC value measures how well the model can distinguish between different classes, with higher values indicating better performance.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"389\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"26.99228791773779%\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"37.53213367609254%\"\u003e\n \u003cp\u003eCross Validation\u003cbr\u003e\u0026nbsp;Accuracy (\u0026plusmn; SD), %\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.47557840616967%\"\u003e\n \u003cp\u003eTest Dataset\u003cbr\u003e\u0026nbsp;AUROC, %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"26.99228791773779%\"\u003e\n \u003cp\u003eVGG16\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"37.53213367609254%\"\u003e\n \u003cp\u003e0.749 \u0026plusmn; 0.110\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.47557840616967%\"\u003e\n \u003cp\u003e95.21\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"26.99228791773779%\"\u003e\n \u003cp\u003eResnet50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"37.53213367609254%\"\u003e\n \u003cp\u003e0.883 \u0026plusmn; 0.120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.47557840616967%\"\u003e\n \u003cp\u003e97.85\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"26.99228791773779%\"\u003e\n \u003cp\u003eInception V3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"37.53213367609254%\"\u003e\n \u003cp\u003e0.897 \u0026plusmn; 0.078\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"35.47557840616967%\"\u003e\n \u003cp\u003e99.32\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eTable 3 Confusion matrix with Inception V3.\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis table shows the performance of the Inception V3 model in classifying data into two categories, RP and Non-RP.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"255\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.294117647058824%\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"56.470588235294116%\" colspan=\"2\"\u003e\n \u003cp\u003ePredicted Label\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.294117647058824%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003eRP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003eNon-RP\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.294117647058824%\" rowspan=\"2\"\u003e\n \u003cp\u003eTrue Label\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003eRP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003e41\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"28.235294117647058%\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"33.333333333333336%\"\u003e\n \u003cp\u003eNo RP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.333333333333336%\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"33.333333333333336%\"\u003e\n \u003cp\u003e23\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cstrong\u003eFigure2\u003c/strong\u003e\u003cstrong\u003e \u003c/strong\u003e\u003cstrong\u003eReceiver operating characteristic (ROC) curve of Inception V3 model with plots of the average of ophthalmologists and a student.\u003c/strong\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eROC curve of Inception V3 model, with performance comparisons to average ophthalmologists and a student, depicted by orange and green triangles respectively.\u003c/p\u003e\n\u003cp\u003eThe horizontal axis shows False Positive Rate (FPR) and the vertical axis shows True Positive Rate (TPR).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 2\u003c/strong\u003e shows the performance of the Inception V3 model along with the ophthalmologists and medical student accuracy. Inception V3 scored best in AUROC compared with two other models shown in \u003cstrong\u003eTable 2\u003c/strong\u003e.\u003c/p\u003e\n\u003ch2\u003e3.3 Visualization\u003c/h2\u003e\n\u003cp\u003eWe conducted a Grad-CAM to visualize which pixels the model sees as important in the prediction process for the model based on Inception V3 to confirm the validity of the model to show that it did not rely on features unrelated to RP.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 3 Grad-CAM analysis of misdiagnosed samples\u003c/strong\u003e. Representative original fundus color images and corresponding Grad-CAM heatmaps are shown for each diagnostic category. The Grad-CAM heatmap show the regions that contributed significantly to the model\u0026apos;s decision: red and yellow indicate high contribution, blue indicates low contribution.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOur analysis revealed that the heatmap predominantly focus on the peripheral regions. \u0026nbsp;However, instances of false positives were observed to be centered around the macular region, indicating potential misfocus. The model correctly identified all RP images. \u003cstrong\u003eFigure 4\u003c/strong\u003e shows two RP images that was misdiagnosed as non-RP by the ophthalmologist but was correctly predicted to be RP by this program. As shown in this example, the program was able to correctly diagnose RP images that would have been missed by an ophthalmologist in a non-RP screening.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFigure 4. RP Images that only AI could correctly diagnose\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eRepresentative original fundus color images and corresponding Grad-CAM heat maps are shown for each diagnostic category.\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003ch2\u003e3.4 Performance comparison with Ophthalmologists and a Medical student\u003c/h2\u003e\n\u003cp\u003e\u0026nbsp;The model and ophthalmologists demonstrated comparable performance in terms of accuracy, recall, specificity, and precision. Additionally, the model outperformed medical students in RP detection. The results are shown in \u003cstrong\u003eTable 4.\u0026nbsp;\u003c/strong\u003e The performance of the model was s comparable to that of ophthalmologists and higher than that of a medical student.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 4 Performance comparison of our AI model, ophthalmologist, and medical student.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe table displays accuracy, recall, specificity, and precision for four ophthalmologists (labeled 1-4), their mean values, a medical student, and a machine learning model.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"676\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003eAccuracy, %\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003eRecall, %\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003eSpecificity, %\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003ePrecision, %\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eOphthalmologist 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e95.45\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e92.68\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eOphthalmologist 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e98.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e97.56\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eOphthalmologist 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e96.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e95.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eOphthalmologist 4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e96.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e95.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eOphthalmologists Mean\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e96.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e95.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eMedical student\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e81.82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e73.17\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e96.77\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"37.813884785819795%\"\u003e\n \u003cp\u003eModel \u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.180206794682423%\"\u003e\n \u003cp\u003e96.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.850812407680944%\"\u003e\n \u003cp\u003e95.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.623338257016249%\"\u003e\n \u003cp\u003e100\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20.53175775480059%\"\u003e\n \u003cp\u003e100 \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"4. Discussion","content":"\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Performance\u003c/h2\u003e \u003cp\u003eRP is a retinal degenerative disease that can lead to loss of sight. Early detection is important in terms of reducing the psychological burden of possible future disabilities even if there is no effective treatment for now. The model using AI can be useful to help ophthalmologists find RP patients. We experimented with VGG16, Resnet50, and Inception V3 to construct the model. As a result, Inception V3 reached the highest accuracy. Since Resnet50 and Inception V3 are improved networks of VGG16, they performed better than VGG16. For an image classification task like this, features can exist anywhere in the image at various sizes. That is why choosing a kernel size is difficult. Inception V3 solves this problem by setting filters of various size at the same level at the same time instead of choosing one. This is probably the reason why Inception V3 could achieve the best performance. In comparison to the prior study conducted by Chen et al., which reported sensitivity and specificity values of 91.2% and 91.71%, respectively, we achieved higher sensitivity and specificity rates in our classification model. This improvement underscores the effectiveness of our approach in accurately identifying individuals with RP, thereby highlighting the potential clinical utility.\u003c/p\u003e \u003cp\u003eOur Grad-Cam analysis, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e3\u003c/span\u003e, revealed that the heatmap predominantly highlights the peripheral regions. This inclination can be rationalized by the typical progression of RP from the periphery inward, leading to more prominent degenerative changes in peripheral areas. Consequently, the focus on these regions appears to be a logical outcome.\u003c/p\u003e \u003cp\u003eIn the two false positive cases, the program primarily concentrated on the central area of the image. The probable cause for this misfocus might arise from the misinterpretation of the macular Tigroid pattern as retinal degenerative changes. Additionally, these errors could stem from insufficient data. These findings underscore the necessity for further exploration into the factors contributing to misfocusing, particularly in distinguishing macular patterns from degenerative changes, to refine the accuracy of the classification model.\u003c/p\u003e \u003cp\u003eIn summary of the visualization, it can be inferred that the model predominantly directs attention to pigmentation in the periphery and the macula for prediction, mirroring the approach of ophthalmologists. This suggests the potential for accurate classification of unknown fundus images.\u003c/p\u003e \u003cp\u003eGoing back to the performance of the model was as high as that of the ophthalmologists\u0026rsquo; diagnoses. The reason for the high performance despite the relatively small amount of data is thought to be that the abnormal findings were relatively easy to detect.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec14\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Limitations and future research\u003c/h2\u003e \u003cp\u003eOur work has several limitations. First, the number of images used in the study was limited, and further accumulation of training data is needed to achieve a practical level. Second, this model only detects the presence or absence of RP and cannot determine the causative gene, stage of the disease, or prognosis. Third, All images used in this study were only taken at Keio University Hospital. To increase the versatility of this model for clinical application, it is necessary to collect data not from a single institution but from multiple institutions.\u003c/p\u003e \u003c/div\u003e"},{"header":"5. Conclusion","content":"\u003cp\u003eThis study is the first to report the development of an AI to diagnose RP from fundus images of Japanese patients. It achieved results comparable to ophthalmologists' diagnoses. Additionally, by utilizing Grad-CAM, we have confirmed that the model prioritizes the peripheral areas, aligning with the known progression pattern of RP. We also discovered that the model may mistakenly identify macular patterns as indicative of RP The results of this study suggest the usefulness of using deep learning for the automatic determination of RP or tools to reduce the burden on ophthalmologists.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eAI; Artificial Intelligence, AUC; Area Under Cover, FPR; False Positive Rate, GradCAM; gradient-weighted class activation mapping, ROC; Receiver Operating Characteristic, RP; retinitis pigmentosa, TPR; True Positive Rat\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors thank the members of the Laboratory of Photobiology, Keio University School of Medicine for their technical and administrative support. Especially, we would like to express our gratitude to those who provide fundus images and who were involved in clinical practice at photographed the data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSaki Ubukata\u003c/strong\u003e: Conceptualization, Methodology, Software, Formal analysis, Investigation, Writing - Original Draft, Visualization\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eKanato Masayoshi:\u003c/strong\u003e Conceptualization, Methodology, Software, Formal analysis, Investigation, Visualization\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eYusaku Katada\u003c/strong\u003e: Conceptualization, Methodology, Resources, Data Curation, Writing - Review \u0026amp; Editing, Funding acquisition\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLizhu Yang:\u0026nbsp;\u003c/strong\u003eMethodology, Resources, Data Curation, Writing - Review \u0026amp; Editing\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNobuhiro Ozawa\u003c/strong\u003e: Resources, Data Curation, Writing - Review \u0026amp; Editing, Funding acquisition\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMari Ibuki\u003c/strong\u003e: Resources, Data Curation, Writing - Review \u0026amp; Editing, Funding acquisition\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eKazuno Negishi\u003c/strong\u003e: Writing - Review \u0026amp; Editing, Supervision\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eToshihide Kurihara\u003c/strong\u003e: Writing - Review \u0026amp; Editing, Supervision\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData availability\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe dataset is not publicly available due to its containing information that could compromise the privacy of research participants but are available from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;The authors declare no competing interests.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eCurr Genomics. 2011;12(4):260\u0026ndash;266.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrado DA, Acosta-Acero M, Maldonado RS. Gene therapy beyond luxturna: a new horizon of the treatment for inherited retinal disease. Curr Opin Ophthalmol. 2020;31(3):147\u0026ndash;154.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiraldi Utz V, Coussa RG, Antaki F, Traboulsi EI. Gene therapy for RPE65-related retinal disease. Ophthalmic Genet. 2018;39(6):671\u0026ndash;677.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTing DSW, Pasquale LR, Peng L, Campbell JP, Lee AY, Raman R, et al. Artificial intelligence and deep learning in ophthalmology. Br J Ophthalmol. 2019;103(2):167\u0026ndash;175.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402\u0026ndash;2410.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen G, Yu M, Li J. Prediction of different eye diseases based on fundus photography via deep transfer learning. J Clin Med. 2021;10(230):5481.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen TC, Lim WS, Wang VY, Ko ML, Chiu SI, Huang YS, Lai F, Yang CM, Hu FR, Jang JSR, Yang CH. Artificial intelligence\u0026ndash;assisted early detection of retinitis pigmentosa\u0026mdash;the most common inherited retinal degeneration. J Digit Imaging. 2021;34(4):\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10278-021-00479-6\u003c/span\u003e\u003cspan address=\"10.1007/s10278-021-00479-6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSelvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22\u0026ndash;29 October 2017; pp. 618\u0026ndash;626.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSimonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://arxiv.org/abs/1409.1556\u003c/span\u003e\u003cspan address=\"https://arxiv.org/abs/1409.1556\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. Accessed 19 Jan 2021.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSzegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015:2818\u0026ndash;2826.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTarg S, Almeida D, Lyman K. Resnet in resnet: generalizing residual architectures. Preprint at \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://arxiv.org/abs/1603.08029\u003c/span\u003e\u003cspan address=\"https://arxiv.org/abs/1603.08029\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDeng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR09), Miami, FL, USA, 20\u0026ndash;25 June 2009.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"retinitis pigmentosa, deep learning, fundus images, artificial intelligence","lastPublishedDoi":"10.21203/rs.3.rs-4851616/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4851616/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eRetinitis pigmentosa (RP) is one group of inherited retinal diseases that are caused by genetic defects that lead to progressive photoreceptor loss and eventual blindness. Early diagnosis will helpful for a effective management of the disease, however, many patients remain unaware of eraly symptoms. Meanwhile, fundus images are widely taken for medical checkups, however, are underused in detecting RP. This study explores the potential of deep learning to identify RP from color fundus images. The dataset contained 200 color fundus images of Japanese RP patients and 121 color fundus images from non-RP subjects from Keio University Hospital. Using transfer learning, pretrained convolutional neural network models -VGG16, Resnet50, and InceptionV3- were finetuned to detect RP. As a result, Inception V3 achieved the best accuracy of 96.97%, which matches the average diagnostic accuracy of ophthalmologists. Using Gradient-weighted Class Activation Mapping (Grad-CAM), we identified peripheral pigmentation in the fundus images as a critical feature for diagnosis, aligning with the known progression patterns of RP. This confirms the robustness and validity of our model, highlighting the utility of deep learning in assisting ophthalmologists with RP screening.\u003c/p\u003e","manuscriptTitle":"Fundus Image Analysis of Retinitis Pigmentosa Using Artificial Intelligence","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-09-09 08:52:03","doi":"10.21203/rs.3.rs-4851616/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"f762dc01-99d2-4d72-8fee-96c6a82208d7","owner":[],"postedDate":"September 9th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":36618534,"name":"Health sciences/Diseases/Eye diseases"},{"id":36618535,"name":"Health sciences/Diseases/Eye diseases/Retinal diseases"}],"tags":[],"updatedAt":"2025-09-05T03:23:35+00:00","versionOfRecord":[],"versionCreatedAt":"2024-09-09 08:52:03","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4851616","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4851616","identity":"rs-4851616","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}