SIENNA: A Generalizable Parameter-Efficient Machine Learning Diagnostic for Clinical Magnetic Resonance Imaging

SIENNA: A Generalizable Parameter-Efficient Machine Learning Diagnostic for Clinical Magnetic Resonance Imaging | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article SIENNA: A Generalizable Parameter-Efficient Machine Learning Diagnostic for Clinical Magnetic Resonance Imaging Janet Paluh, Sreya Sunil, Rahul Rajeev, Ayan Chatterjee, Julie Pilitsis, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4087784/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Contemporary machine learning models for computer vision, although abundant, are largely inappropriate for clinical diagnostics. Clinical sophistication must address data consistency, avoid large parametric needs to reduce model complexity, and achieve stable generalizability across new patient data. Here, we achieve these goals in SIENNA a “Lightweight Energy-efficient Adaptive Next generation” artificial intelligence (LEAN AI) platform along with development of new algorithms for DICOM data consistency and approaches for improved integration of clinical data with deep learning architectures. Applied in the context of brain tumor diagnostics, SIENNA is a nimble AI that requires 175K-285K trainable parameters, 122X less in comparison to other state-of-the-art AI ML tumor models, while outperforming these models. SIENNA is generalizable across diverse patient datasets in inductive tests on benchmark and clinical datasets, achieving high average accuracies of 93–96% in three-way multiclass classification of MRI tumor data, across mixed 1.5 and 3.0 Tesla data and machines. We apply no DICOM MRI data preprocessing beyond data consistency while achieving a parameter-efficient generalizable ML pipeline. SIENNA demonstrates that small clinical datasets can be sufficient to design robust clinical ready architectures to facilitate expanded ML applications in multimodal data integration in a wider range of clinical diagnostic tasks. Health sciences/Medical research/Translational research Biological sciences/Computational biology and bioinformatics/Computational neuroscience/Learning algorithms DICOM convolutional neural network (CNN) adversarial training shortcut learning overfitting clinical diagnostics Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Full Text Additional Declarations Yes there is potential Competing Interest. SIENNA is patent pending in 63,465,719, an Artificial Intelligence System for Tumor Diagnostics of Clinical MRI Datasets. JLP, AM and AC are cofounders of the startup ITrakNeuro Inc. Dr. Pilitsis receives grant support from Medtronic, Boston Scientific, Abbott, NIH 2R01CA166379, NIH R01EB030324, and NIH U44NS115111. She is the medical advisor for Aim Medical Robotics and has stock equity. Supplementary Files SIENNAsupplementalFigsandTables.pdf Figures and Tables Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4087784","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":301835516,"identity":"858500ba-7b45-4874-bb57-3f29464d2e08","order_by":0,"name":"Janet Paluh","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAz0lEQVRIiWNgGAWjYBACxgYGxgOMDTYMfBJwAcJaGIBa0hjYiNYCAkAth0nQwjy7+cFh3h3n5dmkm5895mGwkd1wgJDD5hwzOMx75rZhm8wxc2MehjRjwlpmJAC1tN1mbJNIMJPmYTicSISW9A9ALefs2yTSvwG1/CdGSw7IlgOJbRI5IFsOEKFlzpmCg3PbkpPbZM6USc4xSDaeSUiL4ez2jQ/ettnZ9ku3b5N4U2En20dQywwkDhOPAQHlICAvgezKH0ToGAWjYBSMgpEHANh4Rm+Ye3dMAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-5988-6075","institution":"University at Albany, College of Nanotechnology,Science and Engineering","correspondingAuthor":true,"prefix":"","firstName":"Janet","middleName":"","lastName":"Paluh","suffix":""},{"id":301835517,"identity":"b68a87c0-f28c-47f5-a108-cd071bd794ae","order_by":1,"name":"Sreya Sunil","email":"","orcid":"","institution":"University at Albany, College of Nanotechnology, Science and Engineering","correspondingAuthor":false,"prefix":"","firstName":"Sreya","middleName":"","lastName":"Sunil","suffix":""},{"id":301835518,"identity":"742f6ea7-5945-4d8b-ae0e-df7eb71c5349","order_by":2,"name":"Rahul Rajeev","email":"","orcid":"","institution":"University at Albany, College of Nanotechnology, Science and Engineering","correspondingAuthor":false,"prefix":"","firstName":"Rahul","middleName":"","lastName":"Rajeev","suffix":""},{"id":301835519,"identity":"70c47262-d54d-42ed-b24a-71487604b635","order_by":3,"name":"Ayan Chatterjee","email":"","orcid":"","institution":"Northeastern University","correspondingAuthor":false,"prefix":"","firstName":"Ayan","middleName":"","lastName":"Chatterjee","suffix":""},{"id":301835520,"identity":"e8280163-b782-4633-9bcf-6c1566281fea","order_by":4,"name":"Julie Pilitsis","email":"","orcid":"","institution":"University of Arizona","correspondingAuthor":false,"prefix":"","firstName":"Julie","middleName":"","lastName":"Pilitsis","suffix":""},{"id":301835521,"identity":"e468b0eb-30b3-47ad-9420-543b48b0b33a","order_by":5,"name":"Amitava Mukherjee","email":"","orcid":"","institution":"Amrita University, Amritapuri","correspondingAuthor":false,"prefix":"","firstName":"Amitava","middleName":"","lastName":"Mukherjee","suffix":""}],"badges":[],"createdAt":"2024-03-12 22:05:16","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4087784/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4087784/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":66330472,"identity":"610782fe-e136-47c1-bbb1-86c58f09ce43","added_by":"auto","created_at":"2024-10-10 13:29:08","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":956162,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSIENNA Data Handling Brings Enhanced Feature Extraction for Tumor Diagnostic Multiclassification. \u003c/strong\u003ea. Small dataset analysis benefits by removal of low information flanking Z-stacks. Data modification techniques utilized for model robustness(right top to bottom). Histogram plot comparison before and after equalization of pixel intensities and contrast sharpening using PREMO; adversarial perturbation with magnitude 0.1 imperceptible to human eyes. b. SSIM index of PREMO compared with pre-existing traditional histogram-based equalization algorithms show an approximate increase of 78%, indicating higher retention of scan complexities. SSIM index takes into consideration structural and textural information to quantify the similarity between un-processed scans and equalized scans. c. Visual representation of PREMO samples, focusing on the localized enhancement of image features.\u003c/p\u003e","description":"","filename":"SIENNAFigs151.png","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/0023f6ad179a805c5f7b9edc.png"},{"id":66330475,"identity":"1ca8b522-2c08-4e26-b355-0e1318ae44a3","added_by":"auto","created_at":"2024-10-10 13:29:08","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":895461,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSIENNA’s Optimized Neural Architecture Utilizing Limited Trained Parameters Improves Generalizability. \u003c/strong\u003ea. SIENNA architecture is a linear CNN deep learning based model optimized via hyperparameter tuning with multiple feature extraction layers that map both local and high-order spatial details of brain images. The architecture design incorporates both spatial and depth dimensions and are color-coded.b. Parameter search spaces of SIENNA’s multi-layer CNN pruned using HYPERAS. c. Original scans alongside adversarial counterparts produced through normalized gradient perturbations scaled by a factor of 0.1 and 0.5, convolved with the gradient of the categorical cross-entropy loss of input scan to produce each adversarial example. Initial subtle alterations that might not be noticeable at lower magnitudes can start to manifest as visible noise or distortions in the scan with increase in magnitude of perturbation. d. Clinical diagnostics requires generalizability that is lost by over-processing data and restricts future applications. The workflow of SIENNA integrates minimal data pre-processing, histogram equalization, and adversarial training in a hyper-parameter-tuned network to generate a range of useful performance metrics.\u003c/p\u003e","description":"","filename":"SIENNAFigs152.png","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/ce96f18eab5d1b21c58c8840.png"},{"id":66330474,"identity":"b2285f76-52df-4d1d-a379-d603f6575638","added_by":"auto","created_at":"2024-10-10 13:29:08","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":298803,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSIENNA’s Personalized Performance Delivers Clinically Relevant Outputs for Multiclassification. \u003c/strong\u003ea. Performance metrics are evaluated across patient-specific cases and organized based on accuracy. Stacked bars identify the proportion of accurate and misdiagnosed classifications, represented as TP (successful identification of positive class), TN (successful identification of negative class), FP (incorrect positive identification), and FN (incorrect negative identification). Patient-wise accuracy for GBM lies between 0.8-1.0%and for MET between 0.75-1.0%, where the accuracy is 1.0 when SIENNA detects no FP or FN. b. Confidence percentages mapped across all scans of a patient Z-stack with selected images shown above for three each representative MET (MP1, MP3, MP5) and GBM (GP3, GP5 and GP8) patients.\u003c/p\u003e","description":"","filename":"SIENNAFigs153.png","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/d9f564ba614f397acddf2c95.png"},{"id":66330729,"identity":"0e61eaea-06a0-40f4-ab08-eefbd7df32e9","added_by":"auto","created_at":"2024-10-10 13:37:08","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":1600100,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSIENNA Captures Discriminative Tumor Classification Features by Grad-CAM Explainability Analysis. \u003c/strong\u003ea. Grad-CAM architecture for explainability. Grad-CAM analysis reveals feature maps of the last convolutional layer to identify discriminative features. In terms of FP or FN, these regions inform on where the model may over-rely in decision output. Early decision capabilities for GBM and MET classification can alert pathologists to scan for primary cancers. Normally this action is delayed due to poor or unreliable MRI data and the need for additional diagnostics. b. Heat map analysis diagrams for GBM. c. Heat map analysis diagrams for MET.\u003c/p\u003e","description":"","filename":"SIENNAFigs154.png","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/6a98d0a3e16d31166dc425ea.png"},{"id":66330473,"identity":"5133e19e-34ac-4d8f-851a-ba74b0abc105","added_by":"auto","created_at":"2024-10-10 13:29:08","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":145037,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSIENNA AGI Integrates Data Handling and Multiclass Diagnostics while sustaining a substantially lower parameter count.\u003c/strong\u003e a. SIENNA not only demonstrates parity with but surpasses the benchmark models— Xception, CNN-SVM and NeuroXAI in binary and multi-classification tasks. The narrow error bar indicates a high degree of precision cross validated across 5 runs. b. Plot visualizes the drastic disparity in parameter count across the SOA models used for our comparative analysis with SIENNA. SIENNA demonstrates superior efficiency with fewer parameters. Conversely, Xception and CNN-SVM, despite their higher parameter counts, exhibit inferior performance compared to SIENNA. The accompanying table delineates the specific parameter counts, reinforcing SIENNA's lean yet effective design. c. Overview of the current and expandable potential of SIENNA in diagnostic pipelines. SIENNA is a high-accuracy companion diagnostic for tumor multiclass classification. SIENNA’s ML architecture and integrated optimized data handling are optimal for the analysis of clinical DICOM MRI data and adaptability to different datasets and diagnostic pipelines. SIENNA also offers AI enhanced learning and training for clinical practitioners.\u003c/p\u003e","description":"","filename":"SIENNAFigs155.png","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/a3e724eac3c54ebc6d2549e0.png"},{"id":66331499,"identity":"79154d05-4c96-4afe-a156-6edf60bc1424","added_by":"auto","created_at":"2024-10-10 13:45:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2000737,"visible":true,"origin":"","legend":"","description":"","filename":"SIENNAmanuscriptSuniletal050224.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1_covered_5a45a1fa-a589-4a58-922e-3227bec72953.pdf"},{"id":66330477,"identity":"0f00bf15-4d77-47ed-a1e4-52bb8a1d7fc6","added_by":"auto","created_at":"2024-10-10 13:29:09","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":3184440,"visible":true,"origin":"","legend":"Figures and Tables","description":"","filename":"SIENNAsupplementalFigsandTables.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4087784/v1/5079880c1bd88f45869f3ed4.pdf"}],"financialInterests":"\u003cb\u003eYes\u003c/b\u003e there is potential Competing Interest.\nSIENNA is patent pending in 63,465,719, an Artificial Intelligence System for Tumor Diagnostics of Clinical MRI Datasets. JLP, AM and AC are cofounders of the startup ITrakNeuro Inc. Dr. Pilitsis receives grant support from Medtronic, Boston Scientific, Abbott, NIH 2R01CA166379, NIH R01EB030324, and NIH U44NS115111. She is the medical advisor for Aim Medical Robotics and has stock equity.","formattedTitle":"SIENNA: A Generalizable Parameter-Efficient Machine Learning Diagnostic for Clinical Magnetic Resonance Imaging","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"DICOM, convolutional neural network (CNN), adversarial training, shortcut learning, overfitting, clinical diagnostics","lastPublishedDoi":"10.21203/rs.3.rs-4087784/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4087784/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eContemporary machine learning models for computer vision, although abundant, are largely inappropriate for clinical diagnostics. Clinical sophistication must address data consistency, avoid large parametric needs to reduce model complexity, and achieve stable generalizability across new patient data. Here, we achieve these goals in SIENNA a \u0026ldquo;Lightweight Energy-efficient Adaptive Next generation\u0026rdquo; artificial intelligence (LEAN AI) platform along with development of new algorithms for DICOM data consistency and approaches for improved integration of clinical data with deep learning architectures. Applied in the context of brain tumor diagnostics, SIENNA is a nimble AI that requires 175K-285K trainable parameters, 122X less in comparison to other state-of-the-art AI ML tumor models, while outperforming these models. SIENNA is generalizable across diverse patient datasets in inductive tests on benchmark and clinical datasets, achieving high average accuracies of 93\u0026ndash;96% in three-way multiclass classification of MRI tumor data, across mixed 1.5 and 3.0 Tesla data and machines. We apply no DICOM MRI data preprocessing beyond data consistency while achieving a parameter-efficient generalizable ML pipeline. SIENNA demonstrates that small clinical datasets can be sufficient to design robust clinical ready architectures to facilitate expanded ML applications in multimodal data integration in a wider range of clinical diagnostic tasks.\u003c/p\u003e","manuscriptTitle":"SIENNA: A Generalizable Parameter-Efficient Machine Learning Diagnostic for Clinical Magnetic Resonance Imaging","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-10 13:29:04","doi":"10.21203/rs.3.rs-4087784/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8247713d-3487-4d58-977e-e9abf48416cb","owner":[],"postedDate":"October 10th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":31843588,"name":"Health sciences/Medical research/Translational research"},{"id":31843589,"name":"Biological sciences/Computational biology and bioinformatics/Computational neuroscience/Learning algorithms"}],"tags":[],"updatedAt":"2024-10-10T13:29:04+00:00","versionOfRecord":[],"versionCreatedAt":"2024-10-10 13:29:04","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4087784","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4087784","identity":"rs-4087784","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}