When Nudges Don’t Budge: A Mixed Methods Study of Why EHR-Based Deprescribing Nudges Failed to Change Provider Behavior

When Nudges Don’t Budge: A Mixed Methods Study of Why EHR-Based Deprescribing Nudges Failed to Change Provider Behavior | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article When Nudges Don’t Budge: A Mixed Methods Study of Why EHR-Based Deprescribing Nudges Failed to Change Provider Behavior Ratnalekha V. N. Viswanadham, Hayley M Belli, Tiffany Rose Martinez, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7466262/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Background De-implementation—reducing low-value or harmful care—is critical but often difficult in practice. Nudges via clinical decision support (CDS) tools in electronic health records aim to promote guideline-concordant care, but their effectiveness is mixed. In a randomized trial, we tested CDS nudges to support deprescribing glycemic medications in older adults, aligned with Choosing Wisely guidelines. Despite prior success elsewhere, the intervention had limited impact. The current study evaluated potential reasons why the EHR-based nudges to encourage guideline-based, relaxed glycemic control for older adults with Type 2 Diabetes were not effective in influencing clinician behavior. Methods We conducted a retrospective cohort analysis of EHR data from 67,412 alerts issued to clinicians, promoting different types of glycemic control, including reducing metformin, switching from non-metformin medications to metformin, and discontinuing medication. Comments left by providers on 779 of those firings were coded and thematically analyzed by two authors. Logistic and multinomial logistic regressions were performed to understand the contexts behind the lack of nudge effectiveness at the alert, encounter, patient, and physician levels. Results Out of 67,412 alerts, providers commented in only 1.15% of cases. When they did, they were about 10.7% more likely to act on the alert, but comments were mostly negative (3.28 times more likely). Feedback highlighted three themes: disagreement with guidelines (most common), poor alert fit in workflow, and patient reluctance to change medications. Logistic regressions showed providers were less likely to act on alerts with multiple triggers and more likely to leave negative comments. Multinomial models linked rejection themes to patient and medication traits, noting less rejection related to workflow in patients with limited life expectancy. Disparities in engagement were found, with female providers, patients, and socially vulnerable individuals less likely to comment. Conclusion These findings highlight barriers to de-implementation via CDS. Provider disagreement, misaligned alerts, and patient resistance hinder effectiveness. Low engagement and negative feedback suggest nudges alone may not change behavior without integration into routines. Engagement variation stresses the need for tailored strategies. Future work should refine nudge design to address complexity, align with provider roles, and include patient-centered approaches. Trial Registration The NYU School of Medicine Institutional Review Board (i17-01308) approved the trial, which has the clinicaltrials.gov ID NCT04181307 (https://clinicaltrials.gov/study/NCT04181307), with date of first record on November 26, 2019. electronic health records behavioral economics implementation science diabetes nudge choosing wisely clinical decision support mixed methods research. Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Contributions to the Literature We demonstrate why doctors often disregard automated prompts to discontinue unnecessary medications, citing disagreements with guidelines, poor timing of alerts, and patient resistance. We found that certain groups of patients and providers are less likely to engage with alerts, raising concerns about equity. Tailoring alerts to provider roles and patient needs may improve impact. We show the value of analyzing provider feedback to improve future implementation strategies. INTRODUCTION The de-implementation of healthcare practices—intentionally discontinuing ineffective or potentially harmful interventions—is essential to health services research, aiming to improve patient outcomes and optimize resource allocation. ( 1 ) Yet, despite its significance, de-implementation remains underutilized and undertheorized in clinical practice. ( 2 ) One main reason is the difficulty. De-implementation behavior is challenging because, in a larger context, convincing people to stop habitual behaviors is challenging. Unlike implementation, which often involves building motivation and creating systems for adopting new practices, de-implementation must overcome deeply embedded habits, institutional norms, and fears of unintended consequences. Getting individuals, especially providers with competing demands and limited time, to stop doing something is often more difficult than encouraging them to start a new behavior. For example, in deprescribing, providers face multiple challenges that lead them to stick with usual care, potentially avoiding deprescribing decisions or failing to deprescribe medications. ( 3 , 4 ) Significant time pressures may limit their ability to thoroughly evaluate the necessity of each medication and engage in detailed discussions about deprescribing with patients. Providers might worry about damaging rapport with patients, especially if the patient perceives deprescribing as a withdrawal of care. Providers may lack confidence in deprescribing guidelines due to concerns about the risks and liabilities associated with de-implementation for themselves and their patients, particularly regarding potential adverse outcomes. ( 5 ) To mitigate these, clinical practice guidelines and educational campaigns have been developed to assuage anxiety and challenges of de-implementation. For example, the American Board of Internal Medicine launched the Choosing Wisely (CW) campaign, which focuses on reducing redundant health tests and treatments to improve healthcare quality. ( 5 – 7 ) The campaign has helped bring national attention to the overuse of certain services and promoted more thoughtful conversations between patients and providers. However, despite widespread dissemination, CW guidelines have not always led to measurable reductions in low-value care. This gap between awareness and behavioral change highlights the limitations of passive information dissemination and underscores the need for more active implementation strategies. ( 8 ) However, because disseminating clinical guidelines through informational channels rarely changes practice behavior, implementation science research has sought more effective ways to raise awareness about de-implementation practices, such as deprescribing. ( 9 ) Nudges involve altering the choice architecture to guide individuals toward specific choices while maintaining their autonomy in decision-making.( 10 ) When applied through clinical decision support (CDS) within electronic health records (EHRs), nudges have become increasingly popular for promoting clinical guidelines. CDS proves effective by offering timely and relevant guidance at the moment of decision-making. ( 11 ) For example, it can assist when a provider enters a medication order or composes a message, seamlessly integrating recommendations into their workflow. ( 12 , 13 ) Implementation science research has therefore sought more effective mechanisms to bridge this gap, particularly in high-volume, fast-paced clinical settings. CDS tools, for example, can flag medication orders that may be inappropriate based on patient characteristics or prompt clinicians with tailored recommendations during chart review, order entry, or patient messaging workflows. While some research has reported that nudging in healthcare can effectively change behavior, a growing realization is emerging that not all interventions achieve the desired impact, prompting a need to explore contexts where nudges fall short. ( 14 , 15 ) Contextual factors such as the timing of the alert, competing clinical priorities, and trust in the CDS system all shape how nudges are received and acted upon. Research has shown that multicomponent interventions targeting clinicians to effectively implement care guidelines change clinical practices, thereby reducing the delivery of low-value patient care. ( 8 ) However, the specific mechanisms by which nudges succeed or fail in EHRs remain poorly understood. A study from our group employed a randomized controlled trial across a large academic health system to assess the effectiveness of nudges via clinical decision support in implementing CW guidelines that promote less aggressive glycemic control in pharmacological therapies for geriatric populations with type 2 diabetes (DM2).( 16 ) ( 17 – 19 ) These guidelines reflect a growing consensus that tight glycemic control in older adults may increase the risk of hypoglycemia and other adverse outcomes, particularly in those with comorbidities, cognitive impairment, or limited life expectancy. Results from the RCT showed that the nudges based on these guidelines did not significantly change provider behavior when deprescribing glycemic control medications for older patients. ( 16 ) Given that nudges have shown mixed effects on changing behavior( 20 ), it is crucial to understand why specific nudges within a study led to null results and what contextual factors contributed to these outcomes. Fortunately, one of the CDS tools integrated into the EHR included an option for providers to provide a reason for their acknowledgment, which offers valuable insights into why providers did not act on the CDS recommendations. Previous literature has shown that incorporating provider feedback in the EHR can significantly enhance CDS design.( 21 ) Moreover, the ability to collect contextual insights at the point of care represents a novel and underused method for evaluating and improving CDS systems within a learning health system. This research aims to understand the limited success of implementing nudges in practice within a large, embedded, pragmatic clinical trial. By systematically examining instances when anticipated behavioral changes failed to materialize during de-implementation, we add to the existing body of knowledge on nudges and healthcare. In doing so, we aim to refine strategies for discontinuing healthcare practices that may compromise patient well-being, enhance the design of CDS interventions, and strengthen the evidence base on when and how nudges can work effectively in real-world clinical settings. METHODS Study Aim, Design, and Setting This study employed a mixed-methods approach to evaluate and interpret clinician comments retrospectively, identifying why clinicians agreed or disagreed with proposed care guidelines for de-prescribing DM2 medications in older adult patients. A mixed-methods approach was necessary to capture both the qualitative insights from provider comments regarding disagreements with the CDS nudge intervention and the quantitative insights into the associations between the CDS and sentiments. This approach enables us to understand not only the effectiveness of the alerts but also the contextual factors that influence provider decisions and barriers to implementation. This study adheres to the GRAMMS checklist for mixed methods research, ensuring rigorous reporting of both the thematic analysis of provider comments and the quantitative analysis of alert firings. (22, 23) The study was conducted within the New York University Langone Health (NYULH) EHR and Epic system. Details about the randomized control trial have been published. (19) In summary, a set of six CDS tools, designed using principles of behavioral economics and developed through a user-centered design process, nudged providers to ease the administration of medications aimed at glycemic control for geriatric patients with DM2. These CDSs served as nudges because they guided provider decision-making without restricting providers’ autonomy regarding patient care (i.e., providers could override the suggestions made by the CDS). One type of clinical decision support implemented in the RCT was a built-in reminder in the Epic EHR system known as an “our practice alert” (OPA). The CW guidelines were promoted within the Epic EHR through nine unique OPAs that were designed based on a combination of recommended pharmacological adjustments to glycemic control (switching from a non-Metformin medication to Metformin, lowering the non-Metformin dose, or lowering the Metformin dose) and the patient’s life expectancy (high, medium, and low). The EHR data recorded whether the provider interacted with the OPA and, if so, the nature of their engagement (e.g., whether the OPA was ignored, if the provider acknowledged the OPA but took no action, or if the provider acknowledged the OPA and followed through with the suggestion). Additionally, a provider who engaged with the OPA could leave comments justifying their choice of action regarding the OPA. An example of such an OPA is shown in Figure 1. Study Population The study population consisted of physicians and advanced practice providers at NYULH between December 2016 and July 2023 who could act upon OPAs in the EHR regarding medication titration for DM2 during patient encounters. Data and Coding Data were pulled from the firing of the OPAs between December 22, 2016, and July 28, 2023, with an SQL query from the EHR’s data warehouse. Although the RCT testing the package of nudges was conducted between March 9, 2020, and September 8, 2021, data were collected from OPA firings from the pilot period before the RCT and after the RCT to account for the acute impact of the COVID-19 pandemic on healthcare delivery, which decreased healthcare utilization. (24) The data of interest from the OPAs were the acknowledgment comments left by providers through the OPAs. Covariates collected include patient-level demographics (age, self-identified race and ethnicity, self-identified sex, and BMI), provider-level variables (self-identified gender, type of provider, and provider specialty), and encounter-level variables (the number of firings of other OPAs during the patient’s encounter with the provider as a proxy of provider busyness, the number of unique encounters, and the type of encounter). Race and ethnicity categorizations of patients were based on the U.S. Census Bureau’s categorizations of race and ethnicity. (25) RVNV and HMB independently coded physician comments and then convened to compare their coding. This led to the collaborative determination of primary themes and associated sub-themes. Comments could have been categorized with more than one main theme. The coded themes were classified as either positive (when providers agreed and acted in accordance with the OPA) or negative (when providers did not agree with the OPA). Statistical Analyses Descriptive statistics were collected for the study population to understand the distributions of patient, provider, OPA, and encounter-level covariates. They were also collected for the types of actions taken with the OPAs and the number of comments within the identified themes and sub-themes. Logistic regression was conducted to evaluate whether there were associations between a provider not responding to an OPA’s characteristics (a representation of provider engagement with the OPA) and encounter characteristics (the context in which OPAs are not responded to). Next, a multinomial logistic test was conducted to determine whether an identified rejection theme (versus an acceptance of the OPA) was associated with the type of glycemic control rejected or a patient’s mortality, given that the glycemic controls can be associated with varying provider preferences for glycemic control and that a provider may feel differently. The null hypothesis for these regressions would be that no significant variation existed between why a provider acknowledged a comment and the medication recommendation. The alternate hypothesis implies that the identified themes for rejecting the nudges were significantly associated with different suggestions. Representativeness checks were conducted to check whether commenting behavior was notable for a particular provider or patient population during the RCT period. First, we used logistic regression to determine whether, for OPAs acknowledged, comments on OPAs were left by a representative population of physicians and a representative population of patients in the study. Second, we used a multinomial test to compare whether the distribution of themes in comments left on OPAs during the RCT period was representative of the distribution of themes in comments left during the entire period of the OPAs’ firings, given that the RCT started during the acute COVID-19 pandemic. If the statistical test results fail to reject the null hypothesis (i.e., no statistically significant coefficients in the regressions), then the population of comments from the RCT period would be comparable to those left outside the RCT period. All coefficients were converted into odds ratios for ease of interpretation. Analyses were conducted in RStudio version 4.3.2. RESULTS Descriptive Statistics Table 1 provides descriptive statistics of the patient, provider, encounter, and OPA characteristics. Of the 67,412 times the OPAs were fired between December 22, 2016, and July 28, 2023, providers left 779 comments (1.15%). Of the 25,045 times the OPAs were fired between March 9, 2020, and September 8, 2021 (the RCT period), providers left 215 comments (0.85%). Emergent Themes from Comments Of the comments provided, 82 providers in 763 encounters acted based on the OPA’s recommendation, while 364 providers in 39132 encounters did not, indicating that providers selectively commented on OPAs. This suggests that while most nudges did not lead to deprescribing behavior, a subset of providers was motivated to justify or explain their clinical reasoning, providing a unique window into real-world decision-making. Three major themes were identified among the reasons providers did not utilize the OPA, with some comments meeting multiple major themes. Table 2 provides counts of emergent themes, frequency statistics, and representative comments from those left in the OPAs during their activation. The most frequent theme in the comments was that the providers disagreed with the CW guidelines (N = 308 comments). This disagreement often reflected clinical judgment that diverged from guideline recommendations, underscoring tensions between standardized guidelines and individualized care. In most cases, the provider would determine that the patient had no symptoms of hypoglycemia or issues associated with tight glycemic control and appeared well (N = 160 comments). These comments reflected a perception that the risks of overtreatment were low or not applicable to the patient in question. The second sub-theme that emerged was that the clinician disagreed with the nudge or guidelines (N = 78 comments). These comments more explicitly questioned the relevance, appropriateness, or utility of the CDS tool itself. The third sub-theme was that the provider had other clinical reasons not to act per the OPA’s suggestion, such as the patient having a comorbidity that would not work with the suggestion (e.g., CKD, anemia, or CAD) or a clinical history of not responding well to the OPA’s suggestion (N = 76 comments). In such cases, clinicians provided nuanced clinical reasoning that accounted for complexities not captured by the CDS logic. Fourth, controlling the patient’s A1C was a secondary objective to their current prescription (N = 9 comments). This indicates that glycemic control may have been deprioritized in favor of managing more immediate or pressing health concerns. The second major theme was that nudges were placed in the wrong location within the clinical workflow, making it difficult for providers to take meaningful action (N = 203 comments). This highlights the importance of aligning CDS delivery with key decision-making moments in clinical care. Most commonly, the patient’s glycemic control was often not the responsibility of the provider who received the OPA (e.g., the patient’s primary care provider or endocrinologist), so the glycemic control was not the provider’s responsibility (N = 125 comments). This misalignment likely contributed to alert fatigue or inaction, as the recipient of the alert lacked the contextual authority or responsibility to act. The other subtheme of perceived incorrect placement of the comments was that a provider would want to await a new lab result to reevaluate a patient’s current prescription. The provider also noted that the discussion should be deferred until the next appointment with the patient, indicating that more time was needed to consider the suggestion (N = 89 comments). These comments reflect temporal misalignment—nudges were delivered before the provider felt they had sufficient information or opportunity to act, suggesting the need for more adaptive or anticipatory CDS timing. The third major theme that emerged was that patients needed encouragement to change their medications (N = 69 comments). In these cases, providers recognized the appropriateness of the CDS recommendation but described barriers rooted in patient preferences or resistance to the recommendation. Representative comments on this theme included “patient refuses to reduce meds,” “Patient has been reluctant to stop his medication despite the guidelines which have been discussed, and he clearly understands; however, since he is tolerating his regimen well with no hypoglycemic episodes, he asks to continue treatment,” and “Patient wants to stay on medication optimize glycemic control.” These reflections highlight the interpersonal nature of deprescribing conversations and the limitations of nudges that do not account for the dynamics of shared decision-making. Association Between Response to OPAs and Characteristics We first ran a logistic regression to evaluate whether there was an association between a provider acting upon an OPA (1/0) and the characteristics of the OPAs and encounters (the context in which OPAs are responded to). We also assessed whether the provider commented on the OPA and whether the valence of the comment (negative versus positive) was related to these characteristics. This analysis aimed to identify structural or contextual features that influence not only the likelihood of providers acting on nudges but also how they engaged with and reacted to the CDS tool. From the intercept, providers had a 0.61% chance of acting through the OPA in an encounter (OR = 6.13E-03, SE = 1.47, 95% CI = [2.89E-03, 1.30E-02]). This low base rate reflects the general rarity of deprescribing behavior in response to the CDS nudges. For every extra OPA fired during an encounter, a provider was 53.1% less likely to take the recommended action (OR = 0.469, SE = 1.14, 95% CI = [0.365, 0.603]). This suggests that a higher volume of nudges may contribute to alert fatigue or reduce the perceived salience of each prompt. When commenting through an OPA, providers were approximately three times more likely to leave a negative comment than a positive one (OR = 3.28, SE = 1.28, 95% CI = [2.04, 5.29]). This asymmetry in comment valence may reflect provider skepticism toward the guideline, frustration with workflow disruptions, or disagreement with the timing or content of the nudge. Other encounter-level characteristics, such as patient age, provider specialty, and visit type, were not significantly associated with the valence of provider action or comment (see ST1), suggesting that broader contextual and design-related features may have had a more substantial impact. Together, these findings highlight how both nudge saturation and provider perceptions may influence the effectiveness of CDS. Odds plots can be found in Figure 2, and computed odds ratios can be found in Supplement Tables (STs) 1 and 2. A multinomial logistic test was conducted to determine whether an identified rejection theme (versus acceptance of the OPA) was associated with the type of glycemic control rejected or a patient’s mortality, based on the premise that provider preferences may vary by clinical scenario. We hypothesized that different nudges (e.g., suggestions to reduce insulin versus discontinue sulfonylureas) might elicit distinct reasons for rejection, reflecting variation in provider decision-making. The null hypothesis was that no significant association exists between the reason a provider rejected a recommendation and the type of glycemic control targeted. The alternative hypothesis was that rejection themes were significantly associated with specific types of medication recommendations, suggesting that how providers respond to CDS is not uniform but influenced by the clinical content of the nudge. Compared to agreeing with the OPA-suggested CW guidelines, providers were 31% more likely to comment that the guidelines need improvement (OR = 1.31, SE = 1.08, 95% CI = [1.13, 1.53]). This suggests that disagreement with the clinical recommendation itself—rather than external factors—was a common reason for rejecting the nudge. However, providers were 62.6% less likely to comment that the OPA was in the wrong place in the workflow than to agree with the guideline (OR = 0.374, SE = 1.16, 95% CI = [0.280, 0.499]). This indicates that perceived misalignment with workflow, while important, was a less frequent basis for provider rejection than guideline disagreement. Commenting about the inappropriate placement of the OPA in the workflow was less likely when the patient’s estimated life expectancy was low (OR = 0.427, SE = 1.28, 95% CI = [0.265, 0.688]). This may reflect a greater willingness to consider deprescribing in patients with limited life expectancy, regardless of the timing of the nudge. If a provider commented that a patient needed nudging, it was less likely when their life expectancy was moderate compared to high (OR = 0.745, SE = 1.11, 95% CI = [0.604, 0.918]). This may suggest that providers anticipate more resistance to deprescribing in patients who appear healthier or are expected to live longer. Results are presented in Figure 3 and ST3. Representativeness Checks We conducted three checks to assess whether the populations engaging with OPAs through comments were demographically representative of the individuals featured in the OPAs. Results are presented in Figure 4 (patients) and Figure 5 (providers) and ST4, 5, 6, and 7. First, we used logistic regression to determine whether comments on OPAs were left by a representative population of physicians and a representative population of patients. If comments were left on an OPA, we evaluated whether the valence of the comment (i.e., whether it was positive or negative) was related to the characteristics of the provider or the patient. Compared to male providers, female providers were 21.9% less likely to leave a comment on an OPA (OR 0.788, SE = 1.08, 95% CI = [0.652, 0.954]). Providers were 59% more likely to leave a negative comment than a positive one (OR = 1.59, SE = 0.18, 95% CI = [1.15, 2.19]), which reinforces earlier findings that commentary tends to reflect disagreement or workflow issues. Specialty providers were significantly more likely to leave comments—and particularly negative comments—than general internal medicine providers, indicating differences in CDS response patterns by clinical context or expertise. Female patients were 41.3% less likely to have comments left on their OPA fires compared to male patients (OR = 0.587, SE = 1.09, 95% CI = [0.492, 0.701]), though not significantly more negative comments (OR = 0.808, SE = 1.25, 95% CI = [0.525, 1.24]). Some racial and ethnic minorities, compared to non-Hispanic/Latino White patients, were more likely to have comments left on their OPA firings. However, whether negative comments were left was not necessarily significantly different. Patients of higher social vulnerability were 80.67% less likely to have a comment left on their OPA firing (OR = 0.193, SE = 1.19, 95% CI = [0.138, 0.270]), and patients who had more encounters with their provider had a slightly less chance of having comments left on their OPA firing (OR = 0.957, SE = 1.01, 95% CI = [0.945, 0.970]), potentially reflecting greater reliance on clinical familiarity over CDS prompts in ongoing care relationships. We used a multinomial logistic regression to compare whether the negative themes (compared to the positive theme of the comments left on OPAs during the RCT period) represented the negative themes of the comments left during the entire period of the OPAs’ firings. The most significant difference was that providers were more likely to indicate the OPA was in the wrong place in the workflow during the RCT period than during the RCT period. Otherwise, no significant differences were detected, and the one significant baseline comparison (indicating a wrong placement in the workflow compared to approval) was consistent with the results seen in the multinomial logistic regression of Figure 3, indicating the stability of the results. Results are presented in Figure 6 and ST8. DISCUSSION We conducted a mixed-methods evaluation to identify potential reasons for the ineffectiveness of nudges within the EHR system in promoting CW guidelines among older adults with DM2 in a large academic health system. We help fill a gap in the existing literature that examines the acceptability of implementing deprescribing interventions.( 26 ) Three major themes from the comments reveal barriers to de-implementation. First, de-implementation guidelines need to be specific and clear. The CW guidelines require additional information to increase confidence in them and facilitate structured decision-making regarding glycemic control while allowing for physician autonomy. For example, more robust alternate measures of identifying glycemic control, including when medications are used for non-DM2 reasons and when comorbidities or contraindications come into play, can help providers understand how CW guidelines apply to their patients. The CW guidelines, which informed the CDS design and were algorithmically integrated into the EHR, need to provide explainability support to increase concordant decision-making by providers. Evidence has shown that individuals are less likely to trust algorithm-based decisions when the algorithmic suggestions go against their decision-making in similar contexts. ( 27 ) Second, despite the provider's efforts to prescribe medications and educate the patient, the comments identified that the patient needs support outside of the provider interaction to be empowered to change their medications. There are active efforts within behavioral economics intervention designs to “boost” patients (i.e., combining empowerment with nudging to promote the motivation underlying action). ( 28 ) This is critical when guidelines are based on, to a patient, the disturbing notion of their mortality. The specific CW guidelines made salient information that the patient could fundamentally disagree with (e.g., specific patient complexities around hypoglycemia for physicians or impending mortality for patients). One possible solution to improve patient confidence could be to nudge patients independently of their providers to promote discussion about best care practices. ( 13 ) Another potential solution to mitigate patient hesitancy would be to integrate shared decision-making tools into the EHR that are tailored to each patient's expertise, such as their lived experiences. These tools can help providers and patients reach agreement on making difficult healthcare decisions and acting on guideline recommendations by providing structured guidance and helping them weigh priorities in healthcare.( 29 ) Third, nudges were in the wrong place within the workflow. One, the OPAs were not fired appropriately within the EHR as per the decision-making for medication prescriptions. Providers who were not the primary point of contact for the patient’s DM2 diagnosis were nudged, leading to inaction. Engineering of CDS to focus on the point-of-contact provider could alleviate alert fatigue and increase the success rate of CDS. Two, the CDS could be easily ignored by the provider. Nudges require thoughtful design and integration into other healthcare implementations to ensure their effectiveness, as alert fatigue and clinician desensitization to CDS can lead to the overlooking of highly important CDS. Especially for CDS focused on de-implementing harmful practices, it is much harder to stop ingrained practices than to introduce new ones, which is a focus of most behavioral economics and implementation science research. ( 30 ) Further research can explore how EHR metadata, which describes the granular interactions with the EHR, can inform CDS design into clinical workflows to get provider attention for intended behavior change.( 31 ) The analyses have limitations. First, the OPAs' acknowledgment rate was low. Second, given that very few providers left comments within the OPAs, statistical significance can be attributed to noise rather than being representative of the provider population. Further analyses of misplaced or underutilized CDS in an EHR could explore multiple contexts, incorporating variables such as inpatient versus ambulatory care and other disease types to develop a unified tool that evaluates problematic CDS. ( 32 ) We also acknowledge that we identified the ineffectiveness of the CW guidelines in one clinical setting, disease, and population. The exact results may not be directly applicable to other clinical settings. Still, the methods and data can contribute to the broader research scopes of nudging effectiveness, health services research, and implementation science. CONCLUSION This study demonstrates that EHR comments on CDS can provide valuable evidence regarding how nudges may not affect meaningful clinical behavior changes. We identified key themes surrounding provider resistance to reminders and assessed whether the lack of responsiveness was related to encounter, CDS, patient, or provider characteristics. These contextual variables inform better tailoring of CDS. We contribute to the implementation science literature by offering a method for CDS evaluation to enhance its implementation for healthcare delivery. Other studies that have implemented nudges within the EHR have shown minimal effects on changing provider behaviors toward patients. The comments shed light on why the nudges in this randomized trial were ineffective at significantly altering prescriber behavior. Abbreviations A1C Hemoglobin A1c ABIM American Board of Internal Medicine CAD coronary artery disease CDS Clinical Decision Support CKD Chronic Kidney Disease CW Choosing Wisely Guidelines DM2 Diabetes mellitus type 2 EHR electronic health record IQR interquartile range NYU New York University OPA Our Practice Alert OR Odds ratio RCT randomized controlled trial SEM standard error of the mean ST supplemental table Declarations Ethics approval and consent to participate The NYU Langone Health Institutional Review Board approved this study. A waiver of informed consent was granted because the study posed minimal risk and involved provider-facing clinical decision support tools embedded in routine care. Patients received standard care, and the tools were directed solely at providers within their usual workflow. Consent for publication N/A Availability of data and materials Deidentified data may be available upon request. Competing interests The authors have no known competing interests. Funding This research was supported by NIH NIA Award R33 AG057382 awarded to MPIs ABT and DMM. HMB was supported in part by NIH 5TL1TR001447. RVNV was supported in part by an HRSA Ruth L. Kirschstein National Research Service Award (NIH-T32HP22238). Authors' contributions Substantial contributions to the conception – RVNV and HMB. Design of the work: RVNV and HMB Acquisition, analysis, OR interpretation of data: RVNV and HMB Creation of new software used in the work – CW, JA Drafting the work or substantively revising it: RVNV and HMB Approved the submitted version of this manuscript: RVNV, HMB, TRM, SBB, ABT, DMM Acknowledgments We thank Judd Anderman and Catherine Pollak for their contributions to the study's design and help with data extraction. References Norton WE, Kennedy AE, Chambers DA. Studying de-implementation in health: an analysis of funded research grants. Implement Sci. 2017;12(1):144. Montini T, Graham ID. Entrenched practices and other biases: unpacking the historical, economic, professional, and social resistance to de-implementation. Implement Sci. 2015;10(1):24. Hahn EE, Munoz-Plaza CE, Lee EA, Luong TQ, Mittman BS, Kanter MH, et al. Patient and Physician Perspectives of Deprescribing Potentially Inappropriate Medications in Older Adults with a History of Falls: a Qualitative Study. J Gen Intern Med. 2021;36(10):3015–22. McDaniel CE, House SA, Ralston SL. Behavioral and Psychological Aspects of the Physician Experience with Deimplementation. Pediatr Qual Saf. 2022;7(1):e524. American Geriatrics Society Expert Panel on Care of Older Adults with, Diabetes M, Moreno G, Mangione CM, Kimbro L, Vaisberg E. Guidelines abstracted from the American Geriatrics Society Guidelines for Improving the Care of Older Adults with Diabetes Mellitus: 2013 update. J Am Geriatr Soc. 2013;61(11):2020–6. AGS. Choosing Wisely: An Initiative of the ABIM Foundation.: American Geriatric Society. 2017 [Available from: http://www.choosingwisely.org/clinician-lists/american-geriatrics-society-medication-to-control-type-2-diabetes/ American Diabetes A. 11. Older Adults. Diabetes Care. 2017;40(Suppl 1):S99–104. Cliff BQ, Avancena ALV, Hirth RA, Lee SD. The Impact of Choosing Wisely Interventions on Low-Value Medical Services: A Systematic Review. Milbank Q. 2021;99(4):1024–58. Forrest D, Hoskins A, Hussey R. Clinical guidelines and their implementation. Postgrad Med J. 1996;72:19–22. Thaler RH, Sunstein CR, Nudge. Improving Decisions About Health, Wealth, and Happiness. New Haven & London: Yale University Press; 2008. p. 304. Shah N, Adusumalli S. Nudges and the meaningful adoption of digital health. Per Med. 2020;17(6):429–33. Grouse CK, Waung MW, Holmgren AJ, Mongan J, Neinstein A, Josephson SA, et al. Behavioral nudges in the electronic health record to reduce waste and misuse: 3 interventions. J Am Med Inf Assoc. 2023;30(3):545–50. Adusumalli S, Aragam G, Patel MA. Nudge Towards Cardiovascular Health: Applications of Behavioral Economics for Primary and Secondary Cardiovascular Prevention. Curr Treat Options Cardiovasc Med. 2020;22(9). Wang SY, Groene O. The effectiveness of behavioral economics-informed interventions on physician behavioral change: A systematic literature review. PLoS ONE. 2020;15(6):e0234149. Cho I, Bates DW. Behavioral Economics Interventions in Clinical Decision Support Systems. Yearb Med Inf. 2018;27(1):114–21. Belli HM, Blecker SB, Anderman J, Wong C, Martinez TR, Viswanadham RVN et al. Nudges in the electronic health record to promote appropriate diabetes management in older adults – results from a large, pragmatic, cluster randomized controlled trial. 2023. Workgroup AGSCW. American Geriatrics Society identifies another five things that healthcare providers and patients should question. J Am Geriatr Soc. 2014;62(5):950–60. Chokshi SK, Belli HM, Troxel AB, Blecker S, Blaum C, Testa P, et al. Designing for implementation: user-centered development and pilot testing of a behavioral economic-inspired electronic health record clinical decision support module. Pilot Feasibility Stud. 2019;5:28. Belli HM, Chokshi SK, Hegde R, Troxel AB, Blecker S, Testa PA, et al. Implementation of a Behavioral Economics Electronic Health Record (BE-EHR) Module to Reduce Overtreatment of Diabetes in Older Adults. J Gen Intern Med. 2020;35(11):3254–61. Maier M, Bartos F, Stanley TD, Shanks DR, Harris AJL, Wagenmakers EJ. No evidence for nudging after adjusting for publication bias. Proc Natl Acad Sci U S A. 2022;119(31):e2200300119. Aaron S, McEvoy DS, Ray S, Hickman TT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inf Assoc. 2019;26(1):37–43. O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57. Roy CM, Bollman EB, Carson LM, Northrop AJ, Jackson EF, Moresky RT. Assessing the indirect effects of COVID-19 on healthcare delivery, utilization and health outcomes: a scoping review. Eur J Public Health. 2021;31(3):634–40. Jensen E, Jones N, Medina L, Perry M, Bolender B, Battle K. Measuring Racial and Ethnicic Diversity for the 2020 Census U.S. Census Bureau2021 [Available from: https://www.census.gov/newsroom/blogs/random-samplings/2021/08/measuring-racial-ethnic-diversity-2020-census.html Chua S, Todd A, Reeve E, Smith SM, Fox J, Elsisi Z, et al. Deprescribing interventions in older adults: An overview of systematic reviews. PLoS ONE. 2024;19(6):e0305215. Liu M, Tang XC, Xia SY, Zhang S, Zhu YT, Meng QY. Algorithm Aversion: Evidence from Ridesharing Drivers. Manage Sci. 2023. Hertwig R, Grüne-Yanoff T. Nudging and Boosting: Steering or Empowering Good Decisions. Perspect Psychol Sci. 2017;12(6):973–86. Westermann GM, Verheij F, Winkens B, Verhulst FC, Van Oort FV. Structured shared decision-making using dialogue and visualization: a randomized controlled trial. Patient Educ Couns. 2013;90(1):74–81. McKay VR, Morshed AB, Brownson RC, Proctor EK, Prusaczyk B. Letting Go: Conceptualizing Intervention De-implementation in Public Health and Social Service Settings. Am J Community Psychol. 2018;62(1–2):189–202. Adler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: A new goldmine for health services research? J Biomed Inf. 2020;101:103343. Liu S, McCoy AB, Peterson JF, Lasko TA, Sittig DF, Nelson SD, et al. Leveraging explainable artificial intelligence to optimize clinical decision support. J Am Med Inf Assoc. 2024;31(4):968–74. Tables Table 1 : Descriptive Statistics of Patients and Providers Based on Comments Left or Not Any Alert Firings Alert Firings Without Acknowledgment Comments Alert Firings With Acknowledgment Comments Alert Firings Without Acknowledgment Comments (During RCT) Alert Firings With Acknowledgment Comments (During RCT) Patient Characteristics Unique Patients 7663 7610 (92.6%) 564 (7.4%) 4308 (56.2%) 197 (2.6%) Female 3986 (52.7%) 3965 (52.7%) 267 (47.7%) 2297 (53.2%) 93 (47.0%) Male 3583 (47.3%) 3552 (47.2%) 293 (52.3%) 2020 (46.8%) 105 (53.0%) Other/Unknown 1 (<0.1%) 1 (<0.1%) 0 (0%) 1 (<0.1%) 0 (0%) Non-Hispanic/Latino White 5339 (69.7%) 5303 (69.7%) 381 (67.6%) 3124 (72.5%) 140 (71.1%) Non-Hispanic/Latino Black 879 (11.5%) 874 (11.5%) 70 (12.4%) 466 (10.8%) 24 (12.2%) Asian 336 (4.38%) 336 (4.42%) 22 (3.9%) 186 (4.32%) 5 (2.54%) Hispanic/Latino 308 (4.02%) 308 (4.05%) 29 (5.14%) 150 (3.48%) 8 (4.06%) Multiple Races 31 (0.405%) 30 (0.394%) 4 (0.709%) 23 (0.534%) 1 (0.508%) Native American/Native Alaskan/Native Hawaiian 15 (0.196%) 15 (0.197%) 0 (0%) 8 (0.186%) 0 (0%) Other Race/Ethnicity 8 (0.104%) 8 (0.105%) 1 (0.177%) 1 (0.0232%) 0 (0%) Unknown 748 (9.76%) 737 (9.68%) 57 (10.1%) 350 (8.12%) 19 (9.64%) Age - years (Mean (SE), Median [IQR]) 81.4 (0.0423) 80.2 [77.2, 84.3] 81.4 (0.0424) 80.2 [77.2, 84.3] 81.8 (0.192) 80.8 [78, 84.6] 82.1 (0.0541) 81.1 [78.1, 85] 82.2 (0.337) 81.4 [78.1, 85] Social Vulnerability Index (Mean (SE),Median [IQR]) 0.518 (0.00246) 0.484 [0.272, 0.798] 0.518 (0.00246) 0.489 [0.272, 0.798] 0.433 (0.011) 0.41 [0.195, 0.646] 0.509 (0.00325) 0.475 [0.249, 0.798] 0.397 (0.0186) 0.382 [0.161, 0.572] Patient BMI (Mean [SE], Median [IQR]) 29.4 (0.422) 28.3 [25.2, 32.3] 29.4 (0.424) 28.3 [25.2, 32.3] 28.8 (0.249) 28 [24.9, 32.3] 29.1 (0.0718) 28.4 [25.2, 32.6] 29 (0.417) 28.6 [25, 32.9] Number of Encounters per Patient (Mean[SE], Median [IQR]) 5.35 (0.0497) 3 [2, 7] 5.38 (0.0498) 3 [2, 7] 7.05 (0.273) 5 [2, 9] 7.26 (0.0765) 5 [3, 9] 7.68 (0.45) 6 [3, 10] Provider Characteristics Unique Providers 365 364 82 275 40 Female 137 (37.5%) 137 (37.5%) 28 (34.1%) 110 (40%) 14 (35%) Male 117 117 31 96 13 Other/Unknown 111 110 23 69 13 Physician (MD or DO) 289 (77.3%) 288 (77.2%) 69 (84.1%) 220 (80.292%) 35 (87.5%) Nurse Practitioner (NP) 65 (17.4%) 65 (17.4%) 10 (12.2%) 40 (14.5985%) 5 (12.5%) Physician Assistant (PA) 18 (4.81%) 18 (4.83%) 3 (3.66%) 13 (4.74453%) 0 (0%) Registered Nurse (RN) 1 (0.267%) 1 (0.268%) 0 (0%) 1 (0.364964%) 0 (0%) Internal Medicine - General 268 (71.7%) 268 (71.8%) 61 (74.4%) 197 (71.9%) 26 (65%) Internal Medicine - Specialty 88 (23.5%) 87 (23.3%) 18 (22%) 66 (24.1%) 13 (32.5%) Other Specialties 16 (4.28%) 16 (4.29%) 2 (2.44%) 10 (3.65%) 1 (2.5%) Surgery 3 (0.802%) 3 (0.804%) 1 (1.22%) 1 (0.365%) 0 (0%) Encounter Characteristics Unique Encounters 39654 39132 763 13791 207 Alerts Fired Per Encounter (Mean [SE], Median [IQR]) 3.05 (0.00834) 3 [2, 4] 3.07 (0.0084) 3 [2, 4] 2.3 (0.0538) 2 [1, 3] 2.85 (0.013) 3 [2, 4] 1.96 (0.0805) 2 [1, 2] Follow-Up Visit 34954 (88.1%) 34501 (88.2%) 657 (86.1%) 12254 (88.9%) 187 (90.3%) New Patient Visit 1126 (2.84%) 1103 (2.82%) 40 (5.24%) 362 (2.62%) 9 (4.35%) Preventive Visit 2588 (6.53%) 2543 (6.5%) 62 (8.13%) 878 (6.37%) 10 (4.83%) Operative Care 188 (0.474%) 188 (0.48%) 3 (0.393%) 48 (0.348%) 0 (0%) Other 386 (0.973%) 386 (0.986%) 0 (0%) 50 (0.363%) 0 (0%) Procedure Visit 203 (0.512%) 202 (0.516%) 1 (0.131%) 67 (0.486%) 1 (0.483%) Sick Visit 207 (0.522%) 207 (0.529%) 0 (0%) 131 (0.95%) 0 (0%) Telemedicine Visit 2 (0.00504%) 2 (0.00511%) 0 (0%) 1 (0.00725%) 0 (0%) Monday 7761 (19.6%) 7650 (19.5%) 167 (21.9%) 2724 (19.8%) 39 (18.8%) Tuesday 9112 (23%) 8992 (23%) 179 (23.5%) 3290 (23.9%) 45 (21.7%) Wednesday 8348 (21.1%) 8250 (21.1%) 158 (20.7%) 2858 (20.7%) 60 (29%) Thursday 8297 (20.9%) 8162 (20.9%) 182 (23.9%) 2841 (20.6%) 44 (21.3%) Friday 5553 (14%) 5498 (14%) 74 (9.7%) 1906 (13.8%) 19 (9.18%) Weekend 582 (1.47%) 579 (1.48%) 3 (0.393%) 172 (1.25%) 0 (0%) Alert Characteristics Number of Alert Firings 67412 66633 779 24830 215 Patient Life Expectancy: High 40329 (60.8%) 39821 (60.7%) 508 (66.5%) 15381 (63.9%) 153 (73.9%) Patient Life Expectancy: Medium 24441 (36.8%) 24204 (36.9%) 237 (31%) 8126 (33.8%) 51 (24.6%) Patient Life Expectancy: Low 1564 (2.36%) 1545 (2.36%) 19 (2.49%) 559 (2.32%) 3 (1.45%) Alert Suggestion: Lower Metformin 35474 (53.5%) 35035 (53.4%) 439 (57.5%) 13015 (54.1%) 133 (64.3%) Alert Suggestion: Switch from Non-Metformin to Metformin 26669 (40.2%) 26395 (40.3%) 274 (35.9%) 9194 (38.2%) 58 (28%) Alert Suggestion: Reduced Non-Metformin 4191 (6.32%) 4140 (6.31%) 51 (6.68%) 1857 (7.72%) 16 (7.73%) No Action Taken Through Alert 67330 66561 769 24793 213 Action Taken Through Alert 82 72 10 37 2 Table 2 : Main Themes, Subthemes, and Representative Comments during the entire OPA Activation Period. Theme The CW guidelines need improvement The patient needs nudging. OPA is in the wrong place in the workflow. Provider utilized OPA advice. Number of Comments (N Comments = 565) 308 (54.5%) 69 (12.2%) 203 (36.0%) 208 (36.8%) Sub-themes No symptoms or doing well, so no reason to take action. (N = 160) Clinician doesn’t agree with nudge or guidelines. (N = 78) Other clinical reason to not take action. (N = 76) A1c is secondary objective (e.g., primary weight loss, etc.) (N = 9) N/A Followed by another clinician (e.g., PCP, endocrinologist) so not their responsibility (N = 125) Awaiting new lab result/will re-evaluate. (N = 89) N/A Representative Comments “patient refuses to reduce meds” “await labs” "patient with hyperglycemia with BS at home 200-400" "look at his glucose ; these prompts are silly" Metformin stopped for now patient will continue Prandin in the interim and slowly taper off doing well with low dose metformin which poses no risk “Patient has been reluctant to stop his medication despite the guidelines which have been discussed and he clearly understands however since he is tolerating his regimen wel l with no hypoglycemic episodes he asks to continue treatment.” “DM followed by PCP” dose lowered to once daily from twice daily. "multiple co-morbities with vascular disease and CKD which require better glycemic control" meds reduced to twice daily await labs "CVD secondary prevention." “Patient wants to stay on medication optimize glycemic control” “Will consider at next appointment” Number of unique providers providing this type of comment 59 27 41 47 Number of unique patients for this type of comment 260 60 150 189 OPA firings (N) High Life Expectancy OPAs 219 46 120 136 Medium Life Expectancy OPAs 80 30 82 65 Low Life Expectancy OPAs 9 3 1 7 OPA Suggestion: Lower Metformin 183 50 96 123 OPA Suggestion: Switch from Non-Metformin to Metformin 103 15 93 70 OPA Suggestion: Reduced Non-Metformin 22 4 13 15 Supplementary Files Additionalfile2.docx Adherence to the GRAMMS and COREQ checklists are found in Additional file 2.docx. NudgeBudgeSupplement.xlsx SUPPLEMENTALTABLES.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Major revision 24 Dec, 2025 Reviewers agreed at journal 15 Sep, 2025 Reviewers invited by journal 14 Sep, 2025 Editor assigned by journal 31 Aug, 2025 First submitted to journal 26 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7466262","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":514882524,"identity":"2b9bddf5-21d8-443c-bec4-bf188beb981a","order_by":0,"name":"Ratnalekha V. N. Viswanadham","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABQUlEQVRIie2RMUvDQBSArwTS5UrXhLT4F14oXCjE9q8kHGTK4KSOdalLdVboj8h04HblwEm8NZAlEugUQXBpMRKvOmgTRUeH++DdvXvvPnjHIaTR/FuOEMBuDz6ORIWP31O1dmbN64YKaCvRHxX0qQj0k+Kdy9t8A/7I63LTyitx4HUFy7exHHh9sXoqkT9M+J4yuKNddwERGS8C0wrnwr1ZRCfuBcvw+Cqi9hJFo4ZiIWpaGIQPPDAgnGWdhMfE6ikFUgwORiJsKv3CtCuofZC5AUGVTRNZEvuV3WOQ8vkFo7qlWNR0MHACadDJAzMLkzQmTo9xVYxVC/G2UhBnAHQE6cNZHs5rmqTrY2fIqBpMvXCpWtfNwcK1XZ5O3ERSvtpU0aFKmP3IJlOQokhVa3i5r3yh/WU74LuiRqPRaH7hDUdZfPQ6ohldAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-9786-2090","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":true,"prefix":"","firstName":"Ratnalekha","middleName":"V. N.","lastName":"Viswanadham","suffix":""},{"id":514882525,"identity":"34ce7e44-527f-4783-b56b-c378473a86a2","order_by":1,"name":"Hayley M Belli","email":"","orcid":"","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Hayley","middleName":"M","lastName":"Belli","suffix":""},{"id":514882526,"identity":"44577e8b-f1e7-4478-bf64-78f766daf391","order_by":2,"name":"Tiffany Rose Martinez","email":"","orcid":"","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Tiffany","middleName":"Rose","lastName":"Martinez","suffix":""},{"id":514882527,"identity":"047c1d23-b1bd-49e7-9506-8c43c3afd092","order_by":3,"name":"Christina Wong","email":"","orcid":"","institution":"NYU Langone Medical Center: NYU Langone Health","correspondingAuthor":false,"prefix":"","firstName":"Christina","middleName":"","lastName":"Wong","suffix":""},{"id":514882528,"identity":"63903230-f04a-420c-af6a-9efc379b8fb3","order_by":4,"name":"Saul Blecker","email":"","orcid":"","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Saul","middleName":"","lastName":"Blecker","suffix":""},{"id":514882529,"identity":"c1ce3961-da33-49e7-a51c-4147d03ecd74","order_by":5,"name":"Andrea B Troxel","email":"","orcid":"","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Andrea","middleName":"B","lastName":"Troxel","suffix":""},{"id":514882530,"identity":"de1a5fca-b7b4-41ef-ba44-dd4feeb86301","order_by":6,"name":"Devin M Mann","email":"","orcid":"","institution":"NYU Grossman School of Medicine: New York University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Devin","middleName":"M","lastName":"Mann","suffix":""}],"badges":[],"createdAt":"2025-08-26 21:52:38","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7466262/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7466262/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":91958474,"identity":"c0b9d6b2-0f05-4cbd-b3bd-58387a8aa627","added_by":"auto","created_at":"2025-09-23 07:37:13","extension":"xml","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14858,"visible":true,"origin":"","legend":"","description":"","filename":"iscmISCMD2500327.xml","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/0a8fc689ac06c6b0b35b44df.xml"},{"id":91958489,"identity":"70826876-732a-417b-92d1-5022f7984f61","added_by":"auto","created_at":"2025-09-23 07:37:18","extension":"xml","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1161,"visible":true,"origin":"","legend":"","description":"","filename":"ISCMD25003276281.go.xml","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/74a235be4af810e262178af2.xml"},{"id":91960113,"identity":"551d500e-257b-40cc-9fb2-ca77c999c4fc","added_by":"auto","created_at":"2025-09-23 07:45:12","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":848,"visible":true,"origin":"","legend":"","description":"","filename":"ISCMD2500327Import.xml","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/1675e43fa3bdbb008f91aeb5.xml"},{"id":91958469,"identity":"eedace59-c077-4199-91f8-9ad108f341b2","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"xml","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":232828,"visible":true,"origin":"","legend":"","description":"","filename":"ISCMD25003270enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/e48cf3ee0e3f9f4eaefb7d00.xml"},{"id":91960115,"identity":"0dfd4b36-47c6-481d-b2cd-3134a3f3d820","added_by":"auto","created_at":"2025-09-23 07:45:12","extension":"pptx","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":2930705,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.pptx","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/db387bdfda31b469e0055709.pptx"},{"id":91958472,"identity":"db42e099-f271-42ad-a4c3-7085b1872d11","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"xml","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":230763,"visible":true,"origin":"","legend":"","description":"","filename":"ISCMD25003270structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/6e1d976fa519e5956933e1c9.xml"},{"id":91958471,"identity":"1bc220a7-ad7e-4397-a4da-6dbc47eed5d4","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"html","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":243956,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/bf7ad5c12e2ecec76477c88a.html"},{"id":91958462,"identity":"e644c7f2-1523-4d55-a1a0-80e16720703c","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":110661,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/e39f83a6a6f0804a25320363.jpg"},{"id":91958488,"identity":"e68acf1f-a289-4a95-8790-94daea5fd3e3","added_by":"auto","created_at":"2025-09-23 07:37:18","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":100218,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/d5a9d9b3d2ffa247eb25b349.jpg"},{"id":91961270,"identity":"a4bd2472-5e0e-4161-a25a-9f3ac8dd6ee2","added_by":"auto","created_at":"2025-09-23 07:53:12","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":93212,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/d404dd6a6461262a8b5f5ae7.jpg"},{"id":91958463,"identity":"1ab50662-0102-4050-8889-cbfc6a9a34de","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":94275,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/cc03293e058ed833b3506db7.jpg"},{"id":91960134,"identity":"2830ff98-c7cb-4cc5-a6df-2e2088e47f83","added_by":"auto","created_at":"2025-09-23 07:45:17","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":87056,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/f16eaa6127fbebc2d99b7ad5.jpg"},{"id":91958470,"identity":"a753d705-c525-4a5e-af49-67c926624fa8","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":47229,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Slide6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/613b0d6b21f3cbb86233c5b0.jpg"},{"id":91963208,"identity":"457377c8-4733-4e9b-bfb7-ced42e5a3437","added_by":"auto","created_at":"2025-09-23 08:01:12","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1493477,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/c7224edd-9ac2-47d3-b291-7ed817666501.pdf"},{"id":91960112,"identity":"1da5e306-91e8-445a-b3d4-d4db119fabb9","added_by":"auto","created_at":"2025-09-23 07:45:12","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":17791,"visible":true,"origin":"","legend":"\u003cp\u003eAdherence to the GRAMMS and COREQ checklists are found in \u003cu\u003eAdditional file 2.docx.\u003c/u\u003e\u003c/p\u003e","description":"","filename":"Additionalfile2.docx","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/d1788a27c145b9aff5373261.docx"},{"id":91958487,"identity":"d3c042a4-3df9-4394-b634-8ac1eacb7207","added_by":"auto","created_at":"2025-09-23 07:37:17","extension":"xlsx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":31490,"visible":true,"origin":"","legend":"","description":"","filename":"NudgeBudgeSupplement.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/c200777ac8ba24ed46682031.xlsx"},{"id":91958465,"identity":"a415af1c-eb30-41bc-a0fb-926b4a037c04","added_by":"auto","created_at":"2025-09-23 07:37:12","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":56100,"visible":true,"origin":"","legend":"","description":"","filename":"SUPPLEMENTALTABLES.docx","url":"https://assets-eu.researchsquare.com/files/rs-7466262/v1/db1c9bdfe1250814796665ec.docx"}],"financialInterests":"","formattedTitle":"When Nudges Don’t Budge: A Mixed Methods Study of Why EHR-Based Deprescribing Nudges Failed to Change Provider Behavior","fulltext":[{"header":"Contributions to the Literature","content":"\u003cul\u003e\n \u003cli\u003eWe demonstrate why doctors often disregard automated prompts to discontinue unnecessary medications, citing disagreements with guidelines, poor timing of alerts, and patient resistance.\u003c/li\u003e\n \u003cli\u003eWe found that certain groups of patients and providers are less likely to engage with alerts, raising concerns about equity.\u003c/li\u003e\n \u003cli\u003eTailoring alerts to provider roles and patient needs may improve impact. We show the value of analyzing provider feedback to improve future implementation strategies.\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"INTRODUCTION","content":"\u003cp\u003eThe de-implementation of healthcare practices\u0026mdash;intentionally discontinuing ineffective or potentially harmful interventions\u0026mdash;is essential to health services research, aiming to improve patient outcomes and optimize resource allocation. (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) Yet, despite its significance, de-implementation remains underutilized and undertheorized in clinical practice. (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) One main reason is the difficulty. De-implementation behavior is challenging because, in a larger context, convincing people to stop habitual behaviors is challenging. Unlike implementation, which often involves building motivation and creating systems for adopting new practices, de-implementation must overcome deeply embedded habits, institutional norms, and fears of unintended consequences. Getting individuals, especially providers with competing demands and limited time, to stop doing something is often more difficult than encouraging them to start a new behavior.\u003c/p\u003e\u003cp\u003eFor example, in deprescribing, providers face multiple challenges that lead them to stick with usual care, potentially avoiding deprescribing decisions or failing to deprescribe medications. (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e) Significant time pressures may limit their ability to thoroughly evaluate the necessity of each medication and engage in detailed discussions about deprescribing with patients. Providers might worry about damaging rapport with patients, especially if the patient perceives deprescribing as a withdrawal of care. Providers may lack confidence in deprescribing guidelines due to concerns about the risks and liabilities associated with de-implementation for themselves and their patients, particularly regarding potential adverse outcomes. (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e)\u003c/p\u003e\u003cp\u003e To mitigate these, clinical practice guidelines and educational campaigns have been developed to assuage anxiety and challenges of de-implementation. For example, the American Board of Internal Medicine launched the Choosing Wisely (CW) campaign, which focuses on reducing redundant health tests and treatments to improve healthcare quality. (\u003cspan additionalcitationids=\"CR6\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e) The campaign has helped bring national attention to the overuse of certain services and promoted more thoughtful conversations between patients and providers. However, despite widespread dissemination, CW guidelines have not always led to measurable reductions in low-value care. This gap between awareness and behavioral change highlights the limitations of passive information dissemination and underscores the need for more active implementation strategies. (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e)\u003c/p\u003e\u003cp\u003e However, because disseminating clinical guidelines through informational channels rarely changes practice behavior, implementation science research has sought more effective ways to raise awareness about de-implementation practices, such as deprescribing. (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e) Nudges involve altering the choice architecture to guide individuals toward specific choices while maintaining their autonomy in decision-making.(\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e) When applied through clinical decision support (CDS) within electronic health records (EHRs), nudges have become increasingly popular for promoting clinical guidelines. CDS proves effective by offering timely and relevant guidance at the moment of decision-making. (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e) For example, it can assist when a provider enters a medication order or composes a message, seamlessly integrating recommendations into their workflow. (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) Implementation science research has therefore sought more effective mechanisms to bridge this gap, particularly in high-volume, fast-paced clinical settings. CDS tools, for example, can flag medication orders that may be inappropriate based on patient characteristics or prompt clinicians with tailored recommendations during chart review, order entry, or patient messaging workflows.\u003c/p\u003e\u003cp\u003eWhile some research has reported that nudging in healthcare can effectively change behavior, a growing realization is emerging that not all interventions achieve the desired impact, prompting a need to explore contexts where nudges fall short. (\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e) Contextual factors such as the timing of the alert, competing clinical priorities, and trust in the CDS system all shape how nudges are received and acted upon. Research has shown that multicomponent interventions targeting clinicians to effectively implement care guidelines change clinical practices, thereby reducing the delivery of low-value patient care. (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e) However, the specific mechanisms by which nudges succeed or fail in EHRs remain poorly understood.\u003c/p\u003e\u003cp\u003e A study from our group employed a randomized controlled trial across a large academic health system to assess the effectiveness of nudges via clinical decision support in implementing CW guidelines that promote less aggressive glycemic control in pharmacological therapies for geriatric populations with type 2 diabetes (DM2).(\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) (\u003cspan additionalcitationids=\"CR18\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e) These guidelines reflect a growing consensus that tight glycemic control in older adults may increase the risk of hypoglycemia and other adverse outcomes, particularly in those with comorbidities, cognitive impairment, or limited life expectancy. Results from the RCT showed that the nudges based on these guidelines did not significantly change provider behavior when deprescribing glycemic control medications for older patients. (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eGiven that nudges have shown mixed effects on changing behavior(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e), it is crucial to understand why specific nudges within a study led to null results and what contextual factors contributed to these outcomes. Fortunately, one of the CDS tools integrated into the EHR included an option for providers to provide a reason for their acknowledgment, which offers valuable insights into why providers did not act on the CDS recommendations. Previous literature has shown that incorporating provider feedback in the EHR can significantly enhance CDS design.(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e) Moreover, the ability to collect contextual insights at the point of care represents a novel and underused method for evaluating and improving CDS systems within a learning health system.\u003c/p\u003e\u003cp\u003eThis research aims to understand the limited success of implementing nudges in practice within a large, embedded, pragmatic clinical trial. By systematically examining instances when anticipated behavioral changes failed to materialize during de-implementation, we add to the existing body of knowledge on nudges and healthcare. In doing so, we aim to refine strategies for discontinuing healthcare practices that may compromise patient well-being, enhance the design of CDS interventions, and strengthen the evidence base on when and how nudges can work effectively in real-world clinical settings.\u003c/p\u003e"},{"header":"METHODS","content":"\u003ch2\u003eStudy Aim, Design, and Setting\u003c/h2\u003e\n\u003cp\u003eThis study employed a mixed-methods approach to evaluate and interpret clinician comments retrospectively, identifying why clinicians agreed or disagreed with proposed care guidelines for de-prescribing DM2 medications in older adult patients. A mixed-methods approach was necessary to capture both the qualitative insights from provider comments regarding disagreements with the CDS nudge intervention and the quantitative insights into the associations between the CDS and sentiments. This approach enables us to understand not only the effectiveness of the alerts but also the contextual factors that influence provider decisions and barriers to implementation. This study adheres to the GRAMMS checklist for mixed methods research, ensuring rigorous reporting of both the thematic analysis of provider comments and the quantitative analysis of alert firings. (22, 23) The study was conducted within the New York University Langone Health (NYULH) EHR and Epic system.\u003c/p\u003e\n\u003cp\u003eDetails about the randomized control trial have been published. (19) In summary, a set of six CDS tools, designed using principles of behavioral economics and developed through a user-centered design process, nudged providers to ease the administration of medications aimed at glycemic control for geriatric patients with DM2. These CDSs served as nudges because they guided provider decision-making without restricting providers\u0026rsquo; autonomy regarding patient care (i.e., providers could override the suggestions made by the CDS). One type of clinical decision support implemented in the RCT was a built-in reminder in the Epic EHR system known as an \u0026ldquo;our practice alert\u0026rdquo; (OPA). The CW guidelines were promoted within the Epic EHR through nine unique OPAs that were designed based on a combination of recommended pharmacological adjustments to glycemic control (switching from a non-Metformin medication to Metformin, lowering the non-Metformin dose, or lowering the Metformin dose) and the patient\u0026rsquo;s life expectancy (high, medium, and low). The EHR data recorded whether the provider interacted with the OPA and, if so, the nature of their engagement (e.g., whether the OPA was ignored, if the provider acknowledged the OPA but took no action, or if the provider acknowledged the OPA and followed through with the suggestion). Additionally, a provider who engaged with the OPA could leave comments justifying their choice of action regarding the OPA. An example of such an OPA is shown in Figure 1.\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 1 HERE\u0026gt;\u003c/p\u003e\n\u003ch2\u003eStudy Population\u003c/h2\u003e\n\u003cp\u003eThe study population consisted of physicians and advanced practice providers at NYULH between December 2016 and July 2023 who could act upon OPAs in the EHR regarding medication titration for DM2 during patient encounters.\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eData and Coding\u003c/h2\u003e\n\u003cp\u003eData were pulled from the firing of the OPAs between December 22, 2016, and July 28, 2023, with an SQL query from the EHR\u0026rsquo;s data warehouse. Although the RCT testing the package of nudges was conducted between March 9, 2020, and September 8, 2021, data were collected from OPA firings from the pilot period before the RCT and after the RCT to account for the acute impact of the COVID-19 pandemic on healthcare delivery, which decreased healthcare utilization. (24) The data of interest from the OPAs were the acknowledgment comments left by providers through the OPAs.\u003c/p\u003e\n\u003cp\u003eCovariates collected include patient-level demographics (age, self-identified race and ethnicity, self-identified sex, and BMI), provider-level variables (self-identified gender, type of provider, and provider specialty), and encounter-level variables (the number of firings of other OPAs during the patient\u0026rsquo;s encounter with the provider as a proxy of provider busyness, the number of unique encounters, and the type of encounter). Race and ethnicity categorizations of patients were based on the U.S. Census Bureau\u0026rsquo;s categorizations of race and ethnicity. (25)\u003c/p\u003e\n\u003cp\u003eRVNV and HMB independently coded physician comments and then convened to compare their coding. This led to the collaborative determination of primary themes and associated sub-themes. Comments could have been categorized with more than one main theme. The coded themes were classified as either positive (when providers agreed and acted in accordance with the OPA) or negative (when providers did not agree with the OPA).\u003c/p\u003e\n\u003ch2\u003eStatistical Analyses\u003c/h2\u003e\n\u003cp\u003eDescriptive statistics were collected for the study population to understand the distributions of patient, provider, OPA, and encounter-level covariates. They were also collected for the types of actions taken with the OPAs and the number of comments within the identified themes and sub-themes.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLogistic regression was conducted to evaluate whether there were associations between a provider not responding to an OPA\u0026rsquo;s characteristics (a representation of provider engagement with the OPA) and encounter characteristics (the context in which OPAs are not responded to). Next, a multinomial logistic test was conducted to determine whether an identified rejection theme (versus an acceptance of the OPA) was associated with the type of glycemic control rejected or a patient\u0026rsquo;s mortality, given that the glycemic controls can be associated with varying provider preferences for glycemic control and that a provider may feel differently. The null hypothesis for these regressions would be that no significant variation existed between why a provider acknowledged a comment and the medication recommendation. The alternate hypothesis implies that the identified themes for rejecting the nudges were significantly associated with different suggestions.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eRepresentativeness checks were conducted to check whether commenting behavior was notable for a particular provider or patient population during the RCT period. First, we used logistic regression to determine whether, for OPAs acknowledged, comments on OPAs were left by a representative population of physicians and a representative population of patients in the study. Second, we used a multinomial test to compare whether the distribution of themes in comments left on OPAs during the RCT period was representative of the distribution of themes in comments left during the entire period of the OPAs\u0026rsquo; firings, given that the RCT started during the acute COVID-19 pandemic. \u0026nbsp;If the statistical test results fail to reject the null hypothesis (i.e., no statistically significant coefficients in the regressions), then the population of comments from the RCT period would be comparable to those left outside the RCT period.\u003c/p\u003e\n\u003cp\u003eAll coefficients were converted into odds ratios for ease of interpretation. Analyses were conducted in RStudio version 4.3.2.\u003c/p\u003e"},{"header":"RESULTS","content":"\u003ch2\u003eDescriptive Statistics\u003c/h2\u003e\n\u003cp\u003eTable 1 provides descriptive statistics of the patient, provider, encounter, and OPA characteristics. Of the 67,412 times the OPAs were fired between December 22, 2016, and July 28, 2023, providers left 779 comments (1.15%). Of the 25,045 times the OPAs were fired between March 9, 2020, and September 8, 2021 (the RCT period), providers left 215 comments (0.85%).\u003c/p\u003e\n\u003ch2\u003eEmergent Themes from Comments\u003c/h2\u003e\n\u003cp\u003eOf the comments provided, 82 providers in 763 encounters acted based on the OPA\u0026rsquo;s recommendation, while 364 providers in 39132 encounters did not, indicating that providers selectively commented on OPAs. This suggests that while most nudges did not lead to deprescribing behavior, a subset of providers was motivated to justify or explain their clinical reasoning, providing a unique window into real-world decision-making. Three major themes were identified among the reasons providers did not utilize the OPA, with some comments meeting multiple major themes. Table 2 provides counts of emergent themes, frequency statistics, and representative comments from those left in the OPAs during their activation.\u003c/p\u003e\n\u003cp\u003eThe most frequent theme in the comments was that the providers disagreed with the CW guidelines (N = 308 comments). This disagreement often reflected clinical judgment that diverged from guideline recommendations, underscoring tensions between standardized guidelines and individualized care. In most cases, the provider would determine that the patient had no symptoms of hypoglycemia or issues associated with tight glycemic control and appeared well (N = 160 comments). These comments reflected a perception that the risks of overtreatment were low or not applicable to the patient in question. The second sub-theme that emerged was that the clinician disagreed with the nudge or guidelines (N = 78 comments). These comments more explicitly questioned the relevance, appropriateness, or utility of the CDS tool itself. The third sub-theme was that the provider had other clinical reasons not to act per the OPA\u0026rsquo;s suggestion, such as the patient having a comorbidity that would not work with the suggestion (e.g., CKD, anemia, or CAD) or a clinical history of not responding well to the OPA\u0026rsquo;s suggestion (N = 76 comments). In such cases, clinicians provided nuanced clinical reasoning that accounted for complexities not captured by the CDS logic. Fourth, controlling the patient\u0026rsquo;s A1C was a secondary objective to their current prescription (N = 9 comments). This indicates that glycemic control may have been deprioritized in favor of managing more immediate or pressing health concerns.\u003c/p\u003e\n\u003cp\u003eThe second major theme was that nudges were placed in the wrong location within the clinical workflow, making it difficult for providers to take meaningful action (N = 203 comments). This highlights the importance of aligning CDS delivery with key decision-making moments in clinical care. Most commonly, the patient\u0026rsquo;s glycemic control was often not the responsibility of the provider who received the OPA (e.g., the patient\u0026rsquo;s primary care provider or endocrinologist), so the glycemic control was not the provider\u0026rsquo;s responsibility (N = 125 comments). This misalignment likely contributed to alert fatigue or inaction, as the recipient of the alert lacked the contextual authority or responsibility to act. The other subtheme of perceived incorrect placement of the comments was that a provider would want to await a new lab result to reevaluate a patient\u0026rsquo;s current prescription. The provider also noted that the discussion should be deferred until the next appointment with the patient, indicating that more time was needed to consider the suggestion (N = 89 comments). These comments reflect temporal misalignment\u0026mdash;nudges were delivered before the provider felt they had sufficient information or opportunity to act, suggesting the need for more adaptive or anticipatory CDS timing.\u003c/p\u003e\n\u003cp\u003eThe third major theme that emerged was that patients needed encouragement to change their medications (N = 69 comments). In these cases, providers recognized the appropriateness of the CDS recommendation but described barriers rooted in patient preferences or resistance to the recommendation. Representative comments on this theme included \u0026ldquo;patient refuses to reduce meds,\u0026rdquo; \u0026ldquo;Patient has been reluctant to stop his medication despite the guidelines which have been discussed, and he \u0026nbsp; clearly understands; however, since he is tolerating his regimen well with no hypoglycemic episodes, he asks to continue treatment,\u0026rdquo; and \u0026ldquo;Patient wants to stay on medication optimize glycemic control.\u0026rdquo; These reflections highlight the interpersonal nature of deprescribing conversations and the limitations of nudges that do not account for the dynamics of shared decision-making.\u003c/p\u003e\n\u003ch2\u003eAssociation Between Response to OPAs and Characteristics\u003c/h2\u003e\n\u003cp\u003eWe first ran a logistic regression to evaluate whether there was an association between a provider acting upon an OPA (1/0) and the characteristics of the OPAs and encounters (the context in which OPAs are responded to). We also assessed whether the provider commented on the OPA and whether the valence of the comment (negative versus positive) was related to these characteristics. This analysis aimed to identify structural or contextual features that influence not only the likelihood of providers acting on nudges but also how they engaged with and reacted to the CDS tool.\u003c/p\u003e\n\u003cp\u003eFrom the intercept, providers had a 0.61% chance of acting through the OPA in an encounter (OR = 6.13E-03, SE = 1.47, 95% CI = [2.89E-03, 1.30E-02]). This low base rate reflects the general rarity of deprescribing behavior in response to the CDS nudges. For every extra OPA fired during an encounter, a provider was 53.1% less likely to take the recommended action (OR = 0.469, SE = 1.14, 95% CI = [0.365, 0.603]). This suggests that a higher volume of nudges may contribute to alert fatigue or reduce the perceived salience of each prompt.\u003c/p\u003e\n\u003cp\u003eWhen commenting through an OPA, providers were approximately three times more likely to leave a negative comment than a positive one (OR = 3.28, SE = 1.28, 95% CI = [2.04, 5.29]). This asymmetry in comment valence may reflect provider skepticism toward the guideline, frustration with workflow disruptions, or disagreement with the timing or content of the nudge. Other encounter-level characteristics, such as patient age, provider specialty, and visit type, were not significantly associated with the valence of provider action or comment (see ST1), suggesting that broader contextual and design-related features may have had a more substantial impact. Together, these findings highlight how both nudge saturation and provider perceptions may influence the effectiveness of CDS.\u003c/p\u003e\n\u003cp\u003eOdds plots can be found in Figure 2, and computed odds ratios can be found in Supplement Tables (STs) 1 and 2.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 2 HERE\u0026gt;\u003c/p\u003e\n\u003cp\u003eA multinomial logistic test was conducted to determine whether an identified rejection theme (versus acceptance of the OPA) was associated with the type of glycemic control rejected or a patient\u0026rsquo;s mortality, based on the premise that provider preferences may vary by clinical scenario. We hypothesized that different nudges (e.g., suggestions to reduce insulin versus discontinue sulfonylureas) might elicit distinct reasons for rejection, reflecting variation in provider decision-making. The null hypothesis was that no significant association exists between the reason a provider rejected a recommendation and the type of glycemic control targeted. The alternative hypothesis was that rejection themes were significantly associated with specific types of medication recommendations, suggesting that how providers respond to CDS is not uniform but influenced by the clinical content of the nudge.\u003c/p\u003e\n\u003cp\u003eCompared to agreeing with the OPA-suggested CW guidelines, providers were 31% more likely to comment that the guidelines need improvement (OR = 1.31, SE = 1.08, 95% CI = [1.13, 1.53]). This suggests that disagreement with the clinical recommendation itself\u0026mdash;rather than external factors\u0026mdash;was a common reason for rejecting the nudge. However, providers were 62.6% less likely to comment that the OPA was in the wrong place in the workflow than to agree with the guideline (OR = 0.374, SE = 1.16, 95% CI = [0.280, 0.499]). This indicates that perceived misalignment with workflow, while important, was a less frequent basis for provider rejection than guideline disagreement. Commenting about the inappropriate placement of the OPA in the workflow was less likely when the patient\u0026rsquo;s estimated life expectancy was low (OR = 0.427, SE = 1.28, 95% CI = [0.265, 0.688]). This may reflect a greater willingness to consider deprescribing in patients with limited life expectancy, regardless of the timing of the nudge. If a provider commented that a patient needed nudging, it was less likely when their life expectancy was moderate compared to high (OR = 0.745, SE = 1.11, 95% CI = [0.604, 0.918]). This may suggest that providers anticipate more resistance to deprescribing in patients who appear healthier or are expected to live longer. Results are presented in Figure 3 and ST3.\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 3 HERE\u0026gt;\u003c/p\u003e\n\u003ch2\u003eRepresentativeness Checks\u003c/h2\u003e\n\u003cp\u003eWe conducted three checks to assess whether the populations engaging with OPAs through comments were demographically representative of the individuals featured in the OPAs. Results are presented in Figure 4 (patients) and Figure 5 (providers) and ST4, 5, 6, and 7. First, we used logistic regression to determine whether comments on OPAs were left by a representative population of physicians and a representative population of patients. If comments were left on an OPA, we evaluated whether the valence of the comment (i.e., whether it was positive or negative) was related to the characteristics of the provider or the patient.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCompared to male providers, female providers were 21.9% less likely to leave a comment on an OPA (OR 0.788, SE = 1.08, 95% CI = [0.652, 0.954]). Providers were 59% more likely to leave a negative comment than a positive one (OR = 1.59, SE = 0.18, 95% CI = [1.15, 2.19]), which reinforces earlier findings that commentary tends to reflect disagreement or workflow issues. Specialty providers were significantly more likely to leave comments\u0026mdash;and particularly negative comments\u0026mdash;than general internal medicine providers, indicating differences in CDS response patterns by clinical context or expertise.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFemale patients were 41.3% less likely to have comments left on their OPA fires compared to male patients (OR = 0.587, SE = 1.09, 95% CI = [0.492, 0.701]), though not significantly more negative comments (OR = 0.808, SE = 1.25, 95% CI = [0.525, 1.24]). Some racial and ethnic minorities, compared to non-Hispanic/Latino White patients, were more likely to have comments left on their OPA firings. However, whether negative comments were left was not necessarily significantly different. Patients of higher social vulnerability were 80.67% less likely to have a comment left on their OPA firing (OR = 0.193, SE = 1.19, 95% CI = [0.138, 0.270]), and patients who had more encounters with their provider had a slightly less chance of having comments left on their OPA firing (OR = 0.957, SE = 1.01, 95% CI = [0.945, 0.970]), potentially reflecting \u003cstrong\u003egreater reliance on clinical familiarity over CDS prompts in ongoing care relationships.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 4 HERE\u0026gt;\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 5 HERE\u0026gt;\u003c/p\u003e\n\u003cp\u003eWe used a multinomial logistic regression to compare whether the negative themes (compared to the positive theme of the comments left on OPAs during the RCT period) represented the negative themes of the comments left during the entire period of the OPAs\u0026rsquo; firings. The most significant difference was that providers were more likely to indicate the OPA was in the wrong place in the workflow during the RCT period than during the RCT period. Otherwise, no significant differences were detected, and the one significant baseline comparison (indicating a wrong placement in the workflow compared to approval) was consistent with the results seen in the multinomial logistic regression of Figure 3, indicating the stability of the results. Results are presented in Figure 6 and ST8.\u003c/p\u003e\n\u003cp\u003e\u0026lt;INSERT FIGURE 6 HERE\u0026gt;\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003e We conducted a mixed-methods evaluation to identify potential reasons for the ineffectiveness of nudges within the EHR system in promoting CW guidelines among older adults with DM2 in a large academic health system. We help fill a gap in the existing literature that examines the acceptability of implementing deprescribing interventions.(\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eThree major themes from the comments reveal barriers to de-implementation. First, de-implementation guidelines need to be specific and clear. The CW guidelines require additional information to increase confidence in them and facilitate structured decision-making regarding glycemic control while allowing for physician autonomy. For example, more robust alternate measures of identifying glycemic control, including when medications are used for non-DM2 reasons and when comorbidities or contraindications come into play, can help providers understand how CW guidelines apply to their patients. The CW guidelines, which informed the CDS design and were algorithmically integrated into the EHR, need to provide explainability support to increase concordant decision-making by providers. Evidence has shown that individuals are less likely to trust algorithm-based decisions when the algorithmic suggestions go against their decision-making in similar contexts. (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eSecond, despite the provider's efforts to prescribe medications and educate the patient, the comments identified that the patient needs support outside of the provider interaction to be empowered to change their medications. There are active efforts within behavioral economics intervention designs to \u0026ldquo;boost\u0026rdquo; patients (i.e., combining empowerment with nudging to promote the motivation underlying action). (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e) This is critical when guidelines are based on, to a patient, the disturbing notion of their mortality. The specific CW guidelines made salient information that the patient could fundamentally disagree with (e.g., specific patient complexities around hypoglycemia for physicians or impending mortality for patients). One possible solution to improve patient confidence could be to nudge patients independently of their providers to promote discussion about best care practices. (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) Another potential solution to mitigate patient hesitancy would be to integrate shared decision-making tools into the EHR that are tailored to each patient's expertise, such as their lived experiences. These tools can help providers and patients reach agreement on making difficult healthcare decisions and acting on guideline recommendations by providing structured guidance and helping them weigh priorities in healthcare.(\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eThird, nudges were in the wrong place within the workflow. One, the OPAs were not fired appropriately within the EHR as per the decision-making for medication prescriptions. Providers who were not the primary point of contact for the patient\u0026rsquo;s DM2 diagnosis were nudged, leading to inaction. Engineering of CDS to focus on the point-of-contact provider could alleviate alert fatigue and increase the success rate of CDS. Two, the CDS could be easily ignored by the provider. Nudges require thoughtful design and integration into other healthcare implementations to ensure their effectiveness, as alert fatigue and clinician desensitization to CDS can lead to the overlooking of highly important CDS. Especially for CDS focused on de-implementing harmful practices, it is much harder to stop ingrained practices than to introduce new ones, which is a focus of most behavioral economics and implementation science research. (\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e) Further research can explore how EHR metadata, which describes the granular interactions with the EHR, can inform CDS design into clinical workflows to get provider attention for intended behavior change.(\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eThe analyses have limitations. First, the OPAs' acknowledgment rate was low. Second, given that very few providers left comments within the OPAs, statistical significance can be attributed to noise rather than being representative of the provider population. Further analyses of misplaced or underutilized CDS in an EHR could explore multiple contexts, incorporating variables such as inpatient versus ambulatory care and other disease types to develop a unified tool that evaluates problematic CDS. (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e) We also acknowledge that we identified the ineffectiveness of the CW guidelines in one clinical setting, disease, and population. The exact results may not be directly applicable to other clinical settings. Still, the methods and data can contribute to the broader research scopes of nudging effectiveness, health services research, and implementation science.\u003c/p\u003e"},{"header":"CONCLUSION","content":"\u003cp\u003eThis study demonstrates that EHR comments on CDS can provide valuable evidence regarding how nudges may not affect meaningful clinical behavior changes. We identified key themes surrounding provider resistance to reminders and assessed whether the lack of responsiveness was related to encounter, CDS, patient, or provider characteristics. These contextual variables inform better tailoring of CDS. We contribute to the implementation science literature by offering a method for CDS evaluation to enhance its implementation for healthcare delivery. Other studies that have implemented nudges within the EHR have shown minimal effects on changing provider behaviors toward patients. The comments shed light on why the nudges in this randomized trial were ineffective at significantly altering prescriber behavior.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eA1C\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eHemoglobin A1c\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eABIM\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAmerican Board of Internal Medicine\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eCAD\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003ecoronary artery disease\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eCDS\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eClinical Decision Support\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eCKD\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eChronic Kidney Disease\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eCW\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eChoosing Wisely Guidelines\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eDM2\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eDiabetes mellitus type 2\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eEHR\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eelectronic health record\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eIQR\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003einterquartile range\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eNYU\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eNew York University\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eOPA\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eOur Practice Alert\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eOR\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eOdds ratio\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eRCT\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003erandomized controlled trial\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eSEM\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003estandard error of the mean\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eST\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003esupplemental table\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eEthics approval and consent to participate\u003c/h2\u003e\n\u003cp\u003eThe NYU Langone Health Institutional Review Board approved this study. A waiver of informed consent was granted because the study posed minimal risk and involved provider-facing clinical decision support tools embedded in routine care. Patients received standard care, and the tools were directed solely at providers within their usual workflow.\u003c/p\u003e\n\u003ch2\u003eConsent for publication\u003c/h2\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003ch2\u003eAvailability of data and materials\u003c/h2\u003e\n\u003cp\u003eDeidentified data may be available upon request.\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u003c/h2\u003e\n\u003cp\u003eThe authors have no known competing interests.\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThis research was supported by NIH NIA Award R33 AG057382 awarded to MPIs ABT and DMM. HMB was supported in part by NIH 5TL1TR001447. RVNV was supported in part by an HRSA Ruth L. Kirschstein National Research Service Award (NIH-T32HP22238).\u003c/p\u003e\n\u003ch2\u003eAuthors\u0026apos; contributions\u003c/h2\u003e\n\u003cul\u003e\n \u003cli\u003eSubstantial contributions to the conception \u0026ndash; RVNV and HMB.\u003c/li\u003e\n \u003cli\u003eDesign of the work: RVNV and HMB\u003c/li\u003e\n \u003cli\u003eAcquisition, analysis, OR interpretation of data: RVNV and HMB\u003c/li\u003e\n \u003cli\u003eCreation of new software used in the work \u0026ndash; CW, JA\u003c/li\u003e\n \u003cli\u003eDrafting the work or substantively revising it: RVNV and HMB\u003c/li\u003e\n \u003cli\u003eApproved the submitted version of this manuscript: RVNV, HMB, TRM, SBB, ABT, DMM\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2\u003eAcknowledgments\u003c/h2\u003e\n\u003cp\u003eWe thank Judd Anderman and Catherine Pollak for their contributions to the study\u0026apos;s design and help with data extraction.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eNorton WE, Kennedy AE, Chambers DA. Studying de-implementation in health: an analysis of funded research grants. Implement Sci. 2017;12(1):144.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMontini T, Graham ID. Entrenched practices and other biases: unpacking the historical, economic, professional, and social resistance to de-implementation. Implement Sci. 2015;10(1):24.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHahn EE, Munoz-Plaza CE, Lee EA, Luong TQ, Mittman BS, Kanter MH, et al. Patient and Physician Perspectives of Deprescribing Potentially Inappropriate Medications in Older Adults with a History of Falls: a Qualitative Study. J Gen Intern Med. 2021;36(10):3015\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcDaniel CE, House SA, Ralston SL. Behavioral and Psychological Aspects of the Physician Experience with Deimplementation. Pediatr Qual Saf. 2022;7(1):e524.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAmerican Geriatrics Society Expert Panel on Care of Older Adults with, Diabetes M, Moreno G, Mangione CM, Kimbro L, Vaisberg E. Guidelines abstracted from the American Geriatrics Society Guidelines for Improving the Care of Older Adults with Diabetes Mellitus: 2013 update. J Am Geriatr Soc. 2013;61(11):2020\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAGS. Choosing Wisely: An Initiative of the ABIM Foundation.: American Geriatric Society. 2017 [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.choosingwisely.org/clinician-lists/american-geriatrics-society-medication-to-control-type-2-diabetes/\u003c/span\u003e\u003cspan address=\"http://www.choosingwisely.org/clinician-lists/american-geriatrics-society-medication-to-control-type-2-diabetes/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAmerican Diabetes A. 11. Older Adults. Diabetes Care. 2017;40(Suppl 1):S99\u0026ndash;104.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCliff BQ, Avancena ALV, Hirth RA, Lee SD. The Impact of Choosing Wisely Interventions on Low-Value Medical Services: A Systematic Review. Milbank Q. 2021;99(4):1024\u0026ndash;58.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eForrest D, Hoskins A, Hussey R. Clinical guidelines and their implementation. Postgrad Med J. 1996;72:19\u0026ndash;22.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eThaler RH, Sunstein CR, Nudge. Improving Decisions About Health, Wealth, and Happiness. New Haven \u0026amp; London: Yale University Press; 2008. p. 304.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eShah N, Adusumalli S. Nudges and the meaningful adoption of digital health. Per Med. 2020;17(6):429\u0026ndash;33.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGrouse CK, Waung MW, Holmgren AJ, Mongan J, Neinstein A, Josephson SA, et al. Behavioral nudges in the electronic health record to reduce waste and misuse: 3 interventions. J Am Med Inf Assoc. 2023;30(3):545\u0026ndash;50.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAdusumalli S, Aragam G, Patel MA. Nudge Towards Cardiovascular Health: Applications of Behavioral Economics for Primary and Secondary Cardiovascular Prevention. Curr Treat Options Cardiovasc Med. 2020;22(9).\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWang SY, Groene O. The effectiveness of behavioral economics-informed interventions on physician behavioral change: A systematic literature review. PLoS ONE. 2020;15(6):e0234149.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCho I, Bates DW. Behavioral Economics Interventions in Clinical Decision Support Systems. Yearb Med Inf. 2018;27(1):114\u0026ndash;21.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBelli HM, Blecker SB, Anderman J, Wong C, Martinez TR, Viswanadham RVN et al. Nudges in the electronic health record to promote appropriate diabetes management in older adults \u0026ndash; results from a large, pragmatic, cluster randomized controlled trial. 2023.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWorkgroup AGSCW. American Geriatrics Society identifies another five things that healthcare providers and patients should question. J Am Geriatr Soc. 2014;62(5):950\u0026ndash;60.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChokshi SK, Belli HM, Troxel AB, Blecker S, Blaum C, Testa P, et al. Designing for implementation: user-centered development and pilot testing of a behavioral economic-inspired electronic health record clinical decision support module. Pilot Feasibility Stud. 2019;5:28.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBelli HM, Chokshi SK, Hegde R, Troxel AB, Blecker S, Testa PA, et al. Implementation of a Behavioral Economics Electronic Health Record (BE-EHR) Module to Reduce Overtreatment of Diabetes in Older Adults. J Gen Intern Med. 2020;35(11):3254\u0026ndash;61.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMaier M, Bartos F, Stanley TD, Shanks DR, Harris AJL, Wagenmakers EJ. No evidence for nudging after adjusting for publication bias. Proc Natl Acad Sci U S A. 2022;119(31):e2200300119.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAaron S, McEvoy DS, Ray S, Hickman TT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inf Assoc. 2019;26(1):37\u0026ndash;43.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eO'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349\u0026ndash;57.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRoy CM, Bollman EB, Carson LM, Northrop AJ, Jackson EF, Moresky RT. Assessing the indirect effects of COVID-19 on healthcare delivery, utilization and health outcomes: a scoping review. Eur J Public Health. 2021;31(3):634\u0026ndash;40.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJensen E, Jones N, Medina L, Perry M, Bolender B, Battle K. Measuring Racial and Ethnicic Diversity for the 2020 Census U.S. Census Bureau2021 [Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.census.gov/newsroom/blogs/random-samplings/2021/08/measuring-racial-ethnic-diversity-2020-census.html\u003c/span\u003e\u003cspan address=\"https://www.census.gov/newsroom/blogs/random-samplings/2021/08/measuring-racial-ethnic-diversity-2020-census.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChua S, Todd A, Reeve E, Smith SM, Fox J, Elsisi Z, et al. Deprescribing interventions in older adults: An overview of systematic reviews. PLoS ONE. 2024;19(6):e0305215.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLiu M, Tang XC, Xia SY, Zhang S, Zhu YT, Meng QY. Algorithm Aversion: Evidence from Ridesharing Drivers. Manage Sci. 2023.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHertwig R, Gr\u0026uuml;ne-Yanoff T. Nudging and Boosting: Steering or Empowering Good Decisions. Perspect Psychol Sci. 2017;12(6):973\u0026ndash;86.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWestermann GM, Verheij F, Winkens B, Verhulst FC, Van Oort FV. Structured shared decision-making using dialogue and visualization: a randomized controlled trial. Patient Educ Couns. 2013;90(1):74\u0026ndash;81.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcKay VR, Morshed AB, Brownson RC, Proctor EK, Prusaczyk B. Letting Go: Conceptualizing Intervention De-implementation in Public Health and Social Service Settings. Am J Community Psychol. 2018;62(1\u0026ndash;2):189\u0026ndash;202.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAdler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: A new goldmine for health services research? J Biomed Inf. 2020;101:103343.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLiu S, McCoy AB, Peterson JF, Lasko TA, Sittig DF, Nelson SD, et al. Leveraging explainable artificial intelligence to optimize clinical decision support. J Am Med Inf Assoc. 2024;31(4):968\u0026ndash;74.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003e\u003cstrong\u003eTable\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e1\u003c/strong\u003e\u003cstrong\u003e:\u003c/strong\u003e Descriptive Statistics of Patients and Providers Based on Comments Left or Not\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"100%\" style=\"margin-right: calc(9%); width: 91%;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003eAny Alert Firings\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003eAlert Firings \u003cu\u003eWithout\u003c/u\u003e Acknowledgment Comments\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003eAlert Firings \u003cu\u003eWith\u003c/u\u003e Acknowledgment Comments\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003eAlert Firings \u003cu\u003eWithout\u003c/u\u003e Acknowledgment Comments (During RCT)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003eAlert Firings \u003cu\u003eWith\u003c/u\u003e Acknowledgment Comments (During RCT)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 100px;\"\u003e\n \u003cp\u003ePatient Characteristics\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eUnique Patients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e7663\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e7610 (92.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e564 (7.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e4308 (56.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e197 (2.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e3986 (52.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e3965 (52.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e267 (47.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2297 (53.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e93 (47.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e3583 (47.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e3552 (47.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e293 (52.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2020 (46.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e105 (53.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eOther/Unknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e1 (\u0026lt;0.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e1 (\u0026lt;0.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0 (0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1 (\u0026lt;0.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0 (0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNon-Hispanic/Latino White\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e5339\u0026nbsp;(69.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e5303\u0026nbsp;(69.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e381\u0026nbsp;(67.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e3124\u0026nbsp;(72.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e140\u0026nbsp;(71.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNon-Hispanic/Latino Black\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e879\u0026nbsp;(11.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e874\u0026nbsp;(11.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e70\u0026nbsp;(12.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e466\u0026nbsp;(10.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e24\u0026nbsp;(12.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAsian\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e336\u0026nbsp;(4.38%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e336\u0026nbsp;(4.42%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e22\u0026nbsp;(3.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e186\u0026nbsp;(4.32%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e5\u0026nbsp;(2.54%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eHispanic/Latino\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e308\u0026nbsp;(4.02%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e308\u0026nbsp;(4.05%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e29\u0026nbsp;(5.14%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e150\u0026nbsp;(3.48%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e8\u0026nbsp;(4.06%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eMultiple\u0026nbsp;Races\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e31\u0026nbsp;(0.405%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e30\u0026nbsp;(0.394%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e4\u0026nbsp;(0.709%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e23\u0026nbsp;(0.534%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.508%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNative\u0026nbsp;American/Native\u0026nbsp;Alaskan/Native\u0026nbsp;Hawaiian\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e15\u0026nbsp;(0.196%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e15\u0026nbsp;(0.197%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e8\u0026nbsp;(0.186%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eOther\u0026nbsp;Race/Ethnicity\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e8\u0026nbsp;(0.104%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e8\u0026nbsp;(0.105%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.177%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.0232%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eUnknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e748\u0026nbsp;(9.76%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e737\u0026nbsp;(9.68%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e57\u0026nbsp;(10.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e350\u0026nbsp;(8.12%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e19\u0026nbsp;(9.64%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAge - years\u003cbr\u003e\u0026nbsp;(Mean (SE), Median [IQR])\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e81.4\u0026nbsp;(0.0423)\u003cbr\u003e\u0026nbsp;80.2 [77.2, 84.3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e81.4\u0026nbsp;(0.0424)\u003cbr\u003e\u0026nbsp;80.2 [77.2, 84.3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e81.8\u0026nbsp;(0.192)\u003cbr\u003e\u0026nbsp;80.8 [78, 84.6]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e82.1\u0026nbsp;(0.0541)\u003cbr\u003e\u0026nbsp;81.1 [78.1, 85]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e82.2\u0026nbsp;(0.337)\u003cbr\u003e\u0026nbsp;81.4 [78.1, 85]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eSocial Vulnerability Index\u0026nbsp;\u003cbr\u003e\u0026nbsp;(Mean (SE),Median [IQR])\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e0.518\u0026nbsp;(0.00246)\u003cbr\u003e\u0026nbsp;0.484 [0.272, 0.798]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e0.518\u0026nbsp;(0.00246)\u003cbr\u003e\u0026nbsp;0.489 [0.272, 0.798]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0.433\u0026nbsp;(0.011)\u003cbr\u003e\u0026nbsp;0.41 [0.195, 0.646]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e0.509\u0026nbsp;(0.00325)\u003cbr\u003e\u0026nbsp;0.475 [0.249, 0.798]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0.397\u0026nbsp;(0.0186)\u003cbr\u003e\u0026nbsp;0.382 [0.161, 0.572]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePatient BMI (Mean [SE],\u0026nbsp;\u003cbr\u003e\u0026nbsp;Median [IQR])\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e29.4\u0026nbsp;(0.422)\u003cbr\u003e\u0026nbsp;28.3 [25.2, 32.3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e29.4\u0026nbsp;(0.424)\u003cbr\u003e\u0026nbsp;28.3 [25.2, 32.3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e28.8\u0026nbsp;(0.249)\u003cbr\u003e\u0026nbsp;28 [24.9, 32.3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e29.1\u0026nbsp;(0.0718)\u003cbr\u003e\u0026nbsp;28.4 [25.2, 32.6]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e29\u0026nbsp;(0.417)\u003cbr\u003e\u0026nbsp;28.6 [25, 32.9]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNumber of Encounters per Patient (Mean[SE], Median [IQR])\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e5.35\u0026nbsp;(0.0497)\u003cbr\u003e\u0026nbsp;3 [2, 7]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e5.38\u0026nbsp;(0.0498)\u003cbr\u003e\u0026nbsp;3 [2, 7]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e7.05\u0026nbsp;(0.273)\u003cbr\u003e\u0026nbsp;5 [2, 9]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e7.26\u0026nbsp;(0.0765)\u003cbr\u003e\u0026nbsp;5 [3, 9]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e7.68\u0026nbsp;(0.45)\u003cbr\u003e\u0026nbsp;6 [3, 10]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 100px;\"\u003e\n \u003cp\u003eProvider Characteristics\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eUnique Providers\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e365\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e364\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e275\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e40\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e137 (37.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e137 (37.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e28 (34.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e110 (40%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e14 (35%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e117\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e117\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e31\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003e\u0026nbsp;Other/Unknown\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e111\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e110\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e69\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePhysician (MD or DO)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e289\u0026nbsp;(77.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e288\u0026nbsp;(77.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e69\u0026nbsp;(84.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e220\u0026nbsp;(80.292%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e35\u0026nbsp;(87.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNurse Practitioner (NP)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e65\u0026nbsp;(17.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e65\u0026nbsp;(17.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e10\u0026nbsp;(12.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e40\u0026nbsp;(14.5985%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e5\u0026nbsp;(12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePhysician Assistant (PA)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e18\u0026nbsp;(4.81%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e18\u0026nbsp;(4.83%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(3.66%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e13\u0026nbsp;(4.74453%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eRegistered Nurse (RN)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.267%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.268%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.364964%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eInternal\u0026nbsp;Medicine\u0026nbsp;-\u0026nbsp;General\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e268\u0026nbsp;(71.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e268\u0026nbsp;(71.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e61\u0026nbsp;(74.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e197\u0026nbsp;(71.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e26\u0026nbsp;(65%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eInternal\u0026nbsp;Medicine\u0026nbsp;-\u0026nbsp;Specialty\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e88\u0026nbsp;(23.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e87\u0026nbsp;(23.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e18\u0026nbsp;(22%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e66\u0026nbsp;(24.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e13\u0026nbsp;(32.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eOther\u0026nbsp;Specialties\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e16\u0026nbsp;(4.28%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e16\u0026nbsp;(4.29%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e2\u0026nbsp;(2.44%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e10\u0026nbsp;(3.65%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(2.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eSurgery\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(0.802%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(0.804%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(1.22%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.365%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 100px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 100px;\"\u003e\n \u003cp\u003eEncounter Characteristics\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eUnique Encounters\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e39654\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e39132\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e763\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e13791\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e207\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAlerts Fired Per Encounter\u003cbr\u003e\u0026nbsp;(Mean [SE], Median [IQR])\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e3.05\u0026nbsp;(0.00834)\u003cbr\u003e\u0026nbsp;3 [2, 4]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e3.07\u0026nbsp;(0.0084)\u003cbr\u003e\u0026nbsp;3 [2, 4]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e2.3\u0026nbsp;(0.0538)\u003cbr\u003e\u0026nbsp;2 [1, 3]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2.85\u0026nbsp;(0.013)\u003cbr\u003e\u0026nbsp;3 [2, 4]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e1.96\u0026nbsp;(0.0805)\u003cbr\u003e\u0026nbsp;2 [1, 2]\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eFollow-Up\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e34954\u0026nbsp;(88.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e34501\u0026nbsp;(88.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e657\u0026nbsp;(86.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e12254\u0026nbsp;(88.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e187\u0026nbsp;(90.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNew\u0026nbsp;Patient\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e1126\u0026nbsp;(2.84%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e1103\u0026nbsp;(2.82%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e40\u0026nbsp;(5.24%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e362\u0026nbsp;(2.62%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e9\u0026nbsp;(4.35%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePreventive\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e2588\u0026nbsp;(6.53%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e2543\u0026nbsp;(6.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e62\u0026nbsp;(8.13%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e878\u0026nbsp;(6.37%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e10\u0026nbsp;(4.83%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eOperative\u0026nbsp;Care\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e188\u0026nbsp;(0.474%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e188\u0026nbsp;(0.48%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(0.393%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e48\u0026nbsp;(0.348%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eOther\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e386\u0026nbsp;(0.973%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e386\u0026nbsp;(0.986%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e50\u0026nbsp;(0.363%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eProcedure\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e203\u0026nbsp;(0.512%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e202\u0026nbsp;(0.516%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.131%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e67\u0026nbsp;(0.486%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.483%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eSick\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e207\u0026nbsp;(0.522%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e207\u0026nbsp;(0.529%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e131\u0026nbsp;(0.95%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eTelemedicine\u0026nbsp;Visit\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e2\u0026nbsp;(0.00504%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e2\u0026nbsp;(0.00511%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1\u0026nbsp;(0.00725%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eMonday\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e7761\u0026nbsp;(19.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e7650\u0026nbsp;(19.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e167\u0026nbsp;(21.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2724\u0026nbsp;(19.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e39\u0026nbsp;(18.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eTuesday\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e9112\u0026nbsp;(23%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e8992\u0026nbsp;(23%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e179\u0026nbsp;(23.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e3290\u0026nbsp;(23.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e45\u0026nbsp;(21.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eWednesday\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e8348\u0026nbsp;(21.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e8250\u0026nbsp;(21.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e158\u0026nbsp;(20.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2858\u0026nbsp;(20.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e60\u0026nbsp;(29%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eThursday\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e8297\u0026nbsp;(20.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e8162\u0026nbsp;(20.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e182\u0026nbsp;(23.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e2841\u0026nbsp;(20.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e44\u0026nbsp;(21.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eFriday\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e5553\u0026nbsp;(14%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e5498\u0026nbsp;(14%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e74\u0026nbsp;(9.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1906\u0026nbsp;(13.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e19\u0026nbsp;(9.18%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eWeekend\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e582\u0026nbsp;(1.47%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e579\u0026nbsp;(1.48%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(0.393%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e172\u0026nbsp;(1.25%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e0\u0026nbsp;(0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"6\" style=\"width: 100px;\"\u003e\n \u003cp\u003eAlert Characteristics\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNumber of Alert Firings\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e67412\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e66633\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e779\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e24830\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e215\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePatient Life Expectancy: High\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e40329\u0026nbsp;(60.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e39821\u0026nbsp;(60.7%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e508\u0026nbsp;(66.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e15381\u0026nbsp;(63.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e153\u0026nbsp;(73.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePatient Life Expectancy: Medium\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e24441\u0026nbsp;(36.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e24204\u0026nbsp;(36.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e237\u0026nbsp;(31%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e8126\u0026nbsp;(33.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e51\u0026nbsp;(24.6%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003ePatient Life Expectancy: Low\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e1564\u0026nbsp;(2.36%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e1545\u0026nbsp;(2.36%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e19\u0026nbsp;(2.49%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e559\u0026nbsp;(2.32%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e3\u0026nbsp;(1.45%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAlert Suggestion: Lower Metformin\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e35474\u0026nbsp;(53.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e35035\u0026nbsp;(53.4%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e439\u0026nbsp;(57.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e13015\u0026nbsp;(54.1%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e133\u0026nbsp;(64.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAlert Suggestion: Switch from Non-Metformin to Metformin\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e26669\u0026nbsp;(40.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e26395\u0026nbsp;(40.3%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e274\u0026nbsp;(35.9%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e9194\u0026nbsp;(38.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e58\u0026nbsp;(28%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAlert Suggestion: Reduced Non-Metformin\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e4191\u0026nbsp;(6.32%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e4140\u0026nbsp;(6.31%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e51\u0026nbsp;(6.68%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e1857\u0026nbsp;(7.72%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e16\u0026nbsp;(7.73%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eNo Action Taken Through Alert\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e67330\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e66561\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e769\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e24793\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e213\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 17.6886%;\"\u003e\n \u003cp\u003eAction Taken Through Alert\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 16.5976%;\"\u003e\n \u003cp\u003e82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 17.1492%;\"\u003e\n \u003cp\u003e72\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 14.8751%;\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 11.9954%;\"\u003e\n \u003cp\u003e37\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 21.1338%;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003e2\u003c/strong\u003e\u003cstrong\u003e:\u003c/strong\u003e Main Themes, Subthemes, and Representative Comments during the entire OPA Activation Period.\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"100%\" style=\"margin-right: calc(17%); width: 83%;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTheme\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eThe CW guidelines need improvement\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eThe patient needs nudging.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eOPA is in the wrong place in the workflow.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e\u003cstrong\u003eProvider utilized OPA advice.\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eNumber of Comments\u0026nbsp;\u003cbr\u003e\u0026nbsp;(N Comments = 565)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e308 (54.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e69 (12.2%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e203 (36.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e208 (36.8%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eSub-themes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003eNo symptoms or doing well, so no reason to take action.\u003c/p\u003e\n \u003cp\u003e(N = 160)\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u0026nbsp;Clinician doesn\u0026rsquo;t agree with nudge or guidelines. (N = 78)\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u0026nbsp;Other clinical reason to not take action. (N = 76)\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u0026nbsp;A1c is secondary objective (e.g., primary weight loss, etc.) (N = 9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003eN/A\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003eFollowed by another clinician (e.g., PCP, endocrinologist) so not their responsibility (N = 125)\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u0026nbsp;Awaiting new lab result/will re-evaluate. (N = 89)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003eN/A\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd rowspan=\"5\" valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eRepresentative Comments\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e\u0026ldquo;patient\u0026nbsp;refuses\u0026nbsp;to\u0026nbsp;reduce\u0026nbsp;meds\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e\u0026ldquo;await labs\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e\u0026quot;patient with hyperglycemia with BS at home 200-400\u0026quot;\u003cbr\u003e\u0026nbsp;\u003cbr\u003e\u0026nbsp;\u0026quot;look at his glucose ; these prompts are silly\u0026quot;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003eMetformin stopped for now patient will continue\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;Prandin in the interim and slowly taper off\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003edoing well with low dose metformin which poses no risk\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e\u0026ldquo;Patient has been reluctant to\u0026nbsp; stop his medication despite the\u0026nbsp; guidelines which have been discussed and he clearly understands however since he is tolerating his\u0026nbsp; regimen wel l with no hypoglycemic episodes he asks to continue treatment.\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e\u0026ldquo;DM followed by PCP\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003edose lowered to once daily from twice daily.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e\u0026quot;multiple co-morbities with vascular disease and CKD which require better glycemic control\u0026quot;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003emeds\u0026nbsp;reduced\u0026nbsp;to\u0026nbsp;twice\u0026nbsp;daily\u0026nbsp;await\u0026nbsp;labs\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e\u0026quot;CVD secondary prevention.\u0026quot;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e\u0026ldquo;Patient\u0026nbsp;wants\u0026nbsp;to\u0026nbsp;stay\u0026nbsp;on\u0026nbsp;medication\u0026nbsp;optimize\u0026nbsp;glycemic\u0026nbsp;control\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e\u0026ldquo;Will\u0026nbsp;consider\u0026nbsp;at\u0026nbsp;next\u0026nbsp;appointment\u0026rdquo;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eNumber of unique providers providing this type of comment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e27\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e41\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e47\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eNumber of unique patients for this type of comment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e260\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e60\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e150\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e189\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eOPA firings (N)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eHigh Life Expectancy OPAs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e219\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e46\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e120\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e136\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eMedium Life Expectancy OPAs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e65\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eLow Life Expectancy OPAs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eOPA Suggestion: Lower Metformin\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e183\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e123\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eOPA Suggestion: Switch from Non-Metformin to Metformin\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e103\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e70\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"bottom\" style=\"width: 14px;\"\u003e\n \u003cp\u003eOPA Suggestion: Reduced Non-Metformin\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 24px;\"\u003e\n \u003cp\u003e22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 16.2601%;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 25.8856%;\"\u003e\n \u003cp\u003e13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"bottom\" style=\"width: 40.0944%;\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"implementation-science-communications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"iscm","sideBox":"Learn more about [Implementation Science Communications](https://implementationsciencecomms.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ISCM/default.aspx","title":"Implementation Science Communications","twitterHandle":"@ImplementSci","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"electronic health records, behavioral economics, implementation science, diabetes, nudge, choosing wisely, clinical decision support, mixed methods research.","lastPublishedDoi":"10.21203/rs.3.rs-7466262/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7466262/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDe-implementation—reducing low-value or harmful care—is critical but often difficult in practice. Nudges via clinical decision support (CDS) tools in electronic health records aim to promote guideline-concordant care, but their effectiveness is mixed. In a randomized trial, we tested CDS nudges to support deprescribing glycemic medications in older adults, aligned with Choosing Wisely guidelines. Despite prior success elsewhere, the intervention had limited impact. The current study evaluated potential reasons why the EHR-based nudges to encourage guideline-based, relaxed glycemic control for older adults with Type 2 Diabetes were not effective in influencing clinician behavior.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe conducted a retrospective cohort analysis of EHR data from 67,412 alerts issued to clinicians, promoting different types of glycemic control, including reducing metformin, switching from non-metformin medications to metformin, and discontinuing medication. Comments left by providers on 779 of those firings were coded and thematically analyzed by two authors. Logistic and multinomial logistic regressions were performed to understand the contexts behind the lack of nudge effectiveness at the alert, encounter, patient, and physician levels.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOut of 67,412 alerts, providers commented in only 1.15% of cases. When they did, they were about 10.7% more likely to act on the alert, but comments were mostly negative (3.28 times more likely). Feedback highlighted three themes: disagreement with guidelines (most common), poor alert fit in workflow, and patient reluctance to change medications. Logistic regressions showed providers were less likely to act on alerts with multiple triggers and more likely to leave negative comments. Multinomial models linked rejection themes to patient and medication traits, noting less rejection related to workflow in patients with limited life expectancy. Disparities in engagement were found, with female providers, patients, and socially vulnerable individuals less likely to comment.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThese findings highlight barriers to de-implementation via CDS. Provider disagreement, misaligned alerts, and patient resistance hinder effectiveness. Low engagement and negative feedback suggest nudges alone may not change behavior without integration into routines. Engagement variation stresses the need for tailored strategies. Future work should refine nudge design to address complexity, align with provider roles, and include patient-centered approaches.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTrial Registration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe NYU School of Medicine Institutional Review Board (i17-01308) approved the trial, which has the clinicaltrials.gov ID NCT04181307 (https://clinicaltrials.gov/study/NCT04181307), with date of first record on November 26, 2019.\u003c/p\u003e","manuscriptTitle":"When Nudges Don’t Budge: A Mixed Methods Study of Why EHR-Based Deprescribing Nudges Failed to Change Provider Behavior","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-23 07:37:07","doi":"10.21203/rs.3.rs-7466262/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2025-12-24T11:24:23+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2025-09-15T10:59:35+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-14T18:11:09+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-01T00:22:10+00:00","index":"","fulltext":""},{"type":"submitted","content":"Implementation Science Communications","date":"2025-08-26T17:52:13+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"implementation-science-communications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"iscm","sideBox":"Learn more about [Implementation Science Communications](https://implementationsciencecomms.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ISCM/default.aspx","title":"Implementation Science Communications","twitterHandle":"@ImplementSci","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2ed33fe7-68c2-432b-aae9-a3f2533e0fe7","owner":[],"postedDate":"September 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-14T16:52:21+00:00","versionOfRecord":[],"versionCreatedAt":"2025-09-23 07:37:07","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7466262","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7466262","identity":"rs-7466262","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}