Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning

Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning David Power, Cathy Burke, Michael G. Madden, Ihsan Ullan This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4571143/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 19 Apr, 2025 Read the published version in Scientific Reports → Version 1 posted 12 You are reading this latest preprint version Abstract Artificial intelligence (AI) has the potential to improve healthcare and patient safety and is currently being adopted across various fields of medicine and healthcare. AI and in particular computer vision (CV) are well suited to the analysis of minimally invasive surgical simulation videos for training and performance improvement. CV techniques have rapidly improved in recent years from accurately recognizing objects, instruments, and gestures to phases of surgery and more recently to remembering past surgical steps. Lack of labeled data is a particular problem in surgery considering its complexity, as human annotation and manual assessment are both expensive in time and cost, and in most cases rely on direct intervention of clinical expertise. In this work, a newly collected simulated laparoscopic surgical dataset (LSPD) is presented that will initiate the research in automating this problem and avoiding manual expert assessments. LSPD statistical analyses are given to show similarities and differences between different expertise levels (on Stack, Bands, and Tower Skills). In addition, a 3-dimensional convolutional neural network (3DCNN) is used to classify the experience level of the surgeons, novices, and trainees and is found to achieve good results at distinguishing these, with F1 score of 0.91 and AUC of 0.92. Health sciences/Health care Health sciences/Medical research Physical sciences/Mathematics and computing Laparoscopic Surgery Automated Assessment Deep Learning 3DCNN Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 19 Apr, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 02 Sep, 2024 Reviews received at journal 30 Aug, 2024 Reviewers agreed at journal 20 Aug, 2024 Reviews received at journal 02 Aug, 2024 Reviews received at journal 30 Jul, 2024 Reviewers agreed at journal 23 Jul, 2024 Reviewers agreed at journal 14 Jul, 2024 Reviewers invited by journal 02 Jul, 2024 Editor assigned by journal 02 Jul, 2024 Editor invited by journal 27 Jun, 2024 Submission checks completed at journal 26 Jun, 2024 First submitted to journal 12 Jun, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4571143","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":326805950,"identity":"0f71d3e0-4ddc-47d3-adfa-27daf74585b9","order_by":0,"name":"David Power","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAp0lEQVRIiWNgGAWjYDCCAxBKDkwmkKLFmHQtiQ1EqQYBvgO8Bx/zVNxL33D88AOGBxVEaJE8wJdszHOmOHfDmTQDhoQzRGgxOMBjJp3blpC74QYP0HltRGv5l5BuANbyj2gtDQkJEC0NRGiRPAz0y59jCYYzgX45kHCMCC18x3sPPpxRkyDPd/zww4c/aojQwsDMg2AfIEYDEPAQVjIKRsEoGAUjHAAAiVo1vcdw6Y8AAAAASUVORK5CYII=","orcid":"","institution":"ASSERT Centre, College of Medicine and Health, University College Cork","correspondingAuthor":true,"prefix":"","firstName":"David","middleName":"","lastName":"Power","suffix":""},{"id":326805952,"identity":"b17c6dc0-ef23-45b2-bf6e-cdae50193e25","order_by":1,"name":"Cathy Burke","email":"","orcid":"","institution":"Cork University Maternity Hospital","correspondingAuthor":false,"prefix":"","firstName":"Cathy","middleName":"","lastName":"Burke","suffix":""},{"id":326805955,"identity":"6fece8fe-1f82-47ba-9909-ea3bd5d7b795","order_by":2,"name":"Michael G. Madden","email":"","orcid":"","institution":"School of Computer Science and Data Science Institute, University of Galway","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"G.","lastName":"Madden","suffix":""},{"id":326805957,"identity":"fcdbf28b-70ac-482e-a3d2-8db14fc3990b","order_by":3,"name":"Ihsan Ullan","email":"","orcid":"","institution":"School of Computer Science and Data Science Institute, University of Galway","correspondingAuthor":false,"prefix":"","firstName":"Ihsan","middleName":"","lastName":"Ullan","suffix":""}],"badges":[],"createdAt":"2024-06-12 14:59:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4571143/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4571143/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-96336-5","type":"published","date":"2025-04-19T15:58:03+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":81051622,"identity":"c4570582-2f7c-4b30-9ef5-2181df092224","added_by":"auto","created_at":"2025-04-21 16:11:14","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1011734,"visible":true,"origin":"","legend":"","description":"","filename":"AutomatedPerformance3.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4571143/v1_covered_bb648c13-94ca-461d-9758-630b051f1fde.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Laparoscopic Surgery, Automated Assessment, Deep Learning, 3DCNN","lastPublishedDoi":"10.21203/rs.3.rs-4571143/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4571143/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Artificial intelligence (AI) has the potential to improve healthcare and patient safety and is currently being adopted across various fields of medicine and healthcare. AI and in particular computer vision (CV) are well suited to the analysis of minimally invasive surgical simulation videos for training and performance improvement. CV techniques have rapidly improved in recent years from accurately recognizing objects, instruments, and gestures to phases of surgery and more recently to remembering past surgical steps. Lack of labeled data is a particular problem in surgery considering its complexity, as human annotation and manual assessment are both expensive in time and cost, and in most cases rely on direct intervention of clinical expertise. In this work, a newly collected simulated laparoscopic surgical dataset (LSPD) is presented that will initiate the research in automating this problem and avoiding manual expert assessments. LSPD statistical analyses are given to show similarities and differences between different expertise levels (on Stack, Bands, and Tower Skills). In addition, a 3-dimensional convolutional neural network (3DCNN) is used to classify the experience level of the surgeons, novices, and trainees and is found to achieve good results at distinguishing these, with F1 score of 0.91 and AUC of 0.92.","manuscriptTitle":"Automated Assessment of Simulated Laparoscopic Surgical Skill Performance using Deep Learning","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-07-17 07:28:49","doi":"10.21203/rs.3.rs-4571143/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-09-02T13:33:20+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-30T05:17:16+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"76395874012639091521501014997989813908","date":"2024-08-20T19:16:33+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-02T04:41:00+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-07-31T02:45:29+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"122736199537507017325847631368261656531","date":"2024-07-23T04:02:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"4277696312552254069903173795843211594","date":"2024-07-14T19:20:28+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-07-02T17:24:51+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-07-02T17:23:45+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-06-27T19:58:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-06-26T05:04:12+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2024-06-12T14:57:26+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6370f1d6-b4d3-4d87-8406-301f9b4dcdc4","owner":[],"postedDate":"July 17th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":34578497,"name":"Health sciences/Health care"},{"id":34578498,"name":"Health sciences/Medical research"},{"id":34578499,"name":"Physical sciences/Mathematics and computing"}],"tags":[],"updatedAt":"2025-04-21T16:10:41+00:00","versionOfRecord":{"articleIdentity":"rs-4571143","link":"https://doi.org/10.1038/s41598-025-96336-5","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-04-19 15:58:03","publishedOnDateReadable":"April 19th, 2025"},"versionCreatedAt":"2024-07-17 07:28:49","video":"","vorDoi":"10.1038/s41598-025-96336-5","vorDoiUrl":"https://doi.org/10.1038/s41598-025-96336-5","workflowStages":[]},"version":"v1","identity":"rs-4571143","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4571143","identity":"rs-4571143","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}