{"paper_id":"21ca1507-6f7d-4ed2-af25-5ca2a831e642","body_text":"Active Guidance in Ultrasound Bladder Scanning Using Reinforcement Learning | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Active Guidance in Ultrasound Bladder Scanning Using Reinforcement Learning Hao-Lun Hsu, Mohsen Zahiri, Gary Li, Rashid Al Mukaddim, Hyeonwoo Lee, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6829946/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 15 Jan, 2026 Read the published version in Scientific Reports → Version 1 posted 12 You are reading this latest preprint version Abstract Accurate measurement of bladder volume is essential for diagnosing urinary retention and voiding dysfunction. However, finding optimal view can be challenging for less experienced operators, potentially leading to suboptimal imaging and potential misdiagnoses. This study proposes an intelligent guidance system leveraging reinforcement learning (RL) to improve the acquisition of ultrasound images in ultrasound bladder scanning procedure. We introduce a novel pipeline that incorporates a practical variant of Deep Q-Networks (DQN), known as Adam LMCDQN, which is theoretically validated within linear Markov Decision Processes. Our system aims to offer real-time, adaptive feedback to operators, improving image quality and consistency. We also present a novel domain-specific reward design for reinforcement learning (RL), incorporating domain knowledge to enhance performance. Our results demonstrate a promising 81 % success rate in reaching target points along the transverse direction and 67 % along the longitudinal direction, significantly outperforming supervised deep learning models, which achieved 58 % and 32 %, respectively. This work is among the first to apply RL in ultrasound guidance for bladder assessment, addressing key challenges in exploration strategies and reward mechanisms to enhance clinical efficacy. Physical sciences/Engineering/Biomedical engineering Health sciences/Health care/Medical imaging/Ultrasonography Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 15 Jan, 2026 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 13 Oct, 2025 Reviews received at journal 03 Sep, 2025 Reviews received at journal 13 Aug, 2025 Reviewers agreed at journal 11 Aug, 2025 Reviewers agreed at journal 10 Aug, 2025 Reviews received at journal 22 Jul, 2025 Reviewers agreed at journal 06 Jul, 2025 Reviewers invited by journal 26 Jun, 2025 Editor assigned by journal 26 Jun, 2025 Editor invited by journal 12 Jun, 2025 Submission checks completed at journal 11 Jun, 2025 First submitted to journal 05 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-6829946\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":false,\"archivedVersions\":[],\"articleType\":\"Article\",\"associatedPublications\":[],\"authors\":[{\"id\":476969437,\"identity\":\"88f803d5-a743-4d6f-8e6f-5810893296af\",\"order_by\":0,\"name\":\"Hao-Lun Hsu\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Duke University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Hao-Lun\",\"middleName\":\"\",\"lastName\":\"Hsu\",\"suffix\":\"\"},{\"id\":476969438,\"identity\":\"77ff1909-5c59-4f6e-8a28-4e0b4824024a\",\"order_by\":1,\"name\":\"Mohsen Zahiri\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/ElEQVRIiWNgGAWjYFACHigtwdjAkFDBkMDGDmSQoOUMUAsz8VqAmLGNIYGBmYAG3fbewx8+7mCw55/d3Pbh4bw7eXzMzG0PfzDYyenisMzszLk0yZlnGJgl7hxsnpG47Vkx0GHtxjwMycZmB3BouZFjxszbxsDGcCOxmSFx2+HENmbGNmkGhgOJ23BrMf78t42BRx6sZQ5Ei+QP/FoMpIG+ljAAa2mAaJHgwaflzBkzyd42CQNDkJaEY89AWoB+McDjl+M9xh9+ttnYy91If8z4o+ZO4vz29mcPf1TYyeHSAgUSMAZYGRsDgwFe5SgApmUUjIJRMApGAQIAAHXuXNQ0oM4IAAAAAElFTkSuQmCC\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Mohsen\",\"middleName\":\"\",\"lastName\":\"Zahiri\",\"suffix\":\"\"},{\"id\":476969439,\"identity\":\"84e89792-768d-4a56-b628-71dc324df3f4\",\"order_by\":2,\"name\":\"Gary Li\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Gary\",\"middleName\":\"\",\"lastName\":\"Li\",\"suffix\":\"\"},{\"id\":476969440,\"identity\":\"6923a9f6-691a-4030-b212-a3843bc09dd6\",\"order_by\":3,\"name\":\"Rashid Al Mukaddim\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Rashid\",\"middleName\":\"Al\",\"lastName\":\"Mukaddim\",\"suffix\":\"\"},{\"id\":476969441,\"identity\":\"19f57b57-b63f-47b7-b23c-f956ab8db7cf\",\"order_by\":4,\"name\":\"Hyeonwoo Lee\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Hyeonwoo\",\"middleName\":\"\",\"lastName\":\"Lee\",\"suffix\":\"\"},{\"id\":476969442,\"identity\":\"8e0dafcf-b773-4e8a-9b34-a46d57db1629\",\"order_by\":5,\"name\":\"Martha Grewe Wilson\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Martha\",\"middleName\":\"Grewe\",\"lastName\":\"Wilson\",\"suffix\":\"\"},{\"id\":476969444,\"identity\":\"f0ca5f21-f4d3-4305-8545-26b9c7fab529\",\"order_by\":6,\"name\":\"Joyce Grube\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Joyce\",\"middleName\":\"\",\"lastName\":\"Grube\",\"suffix\":\"\"},{\"id\":476969447,\"identity\":\"d12b6de2-a26a-4823-99f0-3d469eb95d9b\",\"order_by\":7,\"name\":\"Stephen Schmidt\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Stephen\",\"middleName\":\"\",\"lastName\":\"Schmidt\",\"suffix\":\"\"},{\"id\":476969448,\"identity\":\"1edc54fe-e0ff-4c46-8d38-d1bb0e1bc05d\",\"order_by\":8,\"name\":\"Goutam Ghoshal\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Goutam\",\"middleName\":\"\",\"lastName\":\"Ghoshal\",\"suffix\":\"\"},{\"id\":476969450,\"identity\":\"7d26f178-bea8-4f28-a8f7-f5720099dfec\",\"order_by\":9,\"name\":\"Balasundar Raju\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Philips North America\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Balasundar\",\"middleName\":\"\",\"lastName\":\"Raju\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-06-05 14:08:26\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-6829946/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-6829946/v1\",\"draftVersion\":[],\"editorialEvents\":[{\"content\":\"https://doi.org/10.1038/s41598-026-35285-z\",\"type\":\"published\",\"date\":\"2026-01-15T16:29:50+00:00\"}],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":100614682,\"identity\":\"3f578fc3-1d85-46ce-be6a-e2ab783eaddb\",\"added_by\":\"auto\",\"created_at\":\"2026-01-19 17:23:19\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":2105161,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"ActiveGuidanceinUltrasoundBladderScanningUsingReinforcementLearning.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6829946/v1_covered_e6a55d2a-3ea7-4eb1-8b40-ee921bebd013.pdf\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Active Guidance in Ultrasound Bladder Scanning Using Reinforcement Learning\",\"fulltext\":[],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":false,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":false,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":true,\"isAuthorSuppliedPdf\":true,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":true,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"scientific-reports\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"scirep\",\"sideBox\":\"Learn more about [Scientific Reports](http://www.nature.com/srep/)\",\"snPcode\":\"\",\"submissionUrl\":\"\",\"title\":\"Scientific Reports\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"Scientific Reports\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true},\"keywords\":\"\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-6829946/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-6829946/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"Accurate measurement of bladder volume is essential for diagnosing urinary retention and voiding dysfunction. However, finding optimal view can be challenging for less experienced operators, potentially leading to suboptimal imaging and potential misdiagnoses. This study proposes an intelligent guidance system leveraging reinforcement learning (RL) to improve the acquisition of ultrasound images in ultrasound bladder scanning procedure. We introduce a novel pipeline that incorporates a practical variant of Deep Q-Networks (DQN), known as Adam LMCDQN, which is theoretically validated within linear Markov Decision Processes. Our system aims to offer real-time, adaptive feedback to operators, improving image quality and consistency. We also present a novel domain-specific reward design for reinforcement learning (RL), incorporating domain knowledge to enhance performance. Our results demonstrate a promising 81 % success rate in reaching target points along the transverse direction and 67 % along the longitudinal direction, significantly outperforming supervised deep learning models, which achieved 58 % and 32 %, respectively. This work is among the first to apply RL in ultrasound guidance for bladder assessment, addressing key challenges in exploration strategies and reward mechanisms to enhance clinical efficacy.\",\"manuscriptTitle\":\"Active Guidance in Ultrasound Bladder Scanning Using Reinforcement Learning\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-06-30 05:37:00\",\"doi\":\"10.21203/rs.3.rs-6829946/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0},{\"type\":\"decision\",\"content\":\"Revision requested\",\"date\":\"2025-10-13T09:00:42+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-09-04T02:16:03+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-08-13T11:43:21+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"212878605125992780968664104824889529656\",\"date\":\"2025-08-11T05:46:50+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"132652480857202555141714370031790020008\",\"date\":\"2025-08-10T11:56:14+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-07-22T08:51:18+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"198485020171538933836052546147843377066\",\"date\":\"2025-07-06T05:24:05+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewersInvited\",\"content\":\"\",\"date\":\"2025-06-26T06:39:34+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorAssigned\",\"content\":\"\",\"date\":\"2025-06-26T06:34:06+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorInvited\",\"content\":\"\",\"date\":\"2025-06-12T22:32:04+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"checksComplete\",\"content\":\"\",\"date\":\"2025-06-11T15:14:41+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"submitted\",\"content\":\"Scientific Reports\",\"date\":\"2025-06-05T14:04:46+00:00\",\"index\":\"\",\"fulltext\":\"\"}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"scientific-reports\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"scirep\",\"sideBox\":\"Learn more about [Scientific Reports](http://www.nature.com/srep/)\",\"snPcode\":\"\",\"submissionUrl\":\"\",\"title\":\"Scientific Reports\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"Scientific Reports\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"a42c3cb1-c86e-4257-8f19-59232586f201\",\"owner\":[],\"postedDate\":\"June 30th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"published-in-journal\",\"subjectAreas\":[{\"id\":50649801,\"name\":\"Physical sciences/Engineering/Biomedical engineering\"},{\"id\":50649802,\"name\":\"Health sciences/Health care/Medical imaging/Ultrasonography\"}],\"tags\":[],\"updatedAt\":\"2026-01-19T16:47:57+00:00\",\"versionOfRecord\":{\"articleIdentity\":\"rs-6829946\",\"link\":\"https://doi.org/10.1038/s41598-026-35285-z\",\"journal\":{\"identity\":\"scientific-reports\",\"isVorOnly\":false,\"title\":\"Scientific Reports\"},\"publishedOn\":\"2026-01-15 16:29:50\",\"publishedOnDateReadable\":\"January 15th, 2026\"},\"versionCreatedAt\":\"2025-06-30 05:37:00\",\"video\":\"\",\"vorDoi\":\"10.1038/s41598-026-35285-z\",\"vorDoiUrl\":\"https://doi.org/10.1038/s41598-026-35285-z\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-6829946\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-6829946\",\"identity\":\"rs-6829946\",\"version\":[\"v1\"]},\"buildId\":\"XKTyCvWXoU3ODBz1xrDgd\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}