Examining Usability of RxFill: Integrating Health IT to Support Medication Adherence

Examining Usability of RxFill: Integrating Health IT to Support Medication Adherence | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Examining Usability of RxFill: Integrating Health IT to Support Medication Adherence Taylor L. Watterson, Aaron M. Gilson, Peter C. Kleinschmidt, Jamie A. Stone This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5485583/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 16 Apr, 2026 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted 8 You are reading this latest preprint version Abstract Background Over 50% of patients do not take their medications as prescribed or are non-adherent. Primary care providers are well positioned to address non-adherence. However, current methods for obtaining adherence are unreliable. RxFill integrates prescription medication fill status and dispense dates from community pharmacies into clinic electronic health records (EHRs). The goal of this study was to examine RxFill usability during a simulated case with primary care providers. Methods The study took place at an academic health system that implemented RxFill in June 2022. Participants were asked to review a simulated patient chart and “think aloud.” After, a semi-structured interview elicited attitudes towards RxFill, including usability. Participants also completed the System Usability Scale (SUS), which scores the functionality on a scale of 0-100. Think-aloud commentary and interview transcripts were analyzed via qualitative content analysis. Results Eight providers participated in the study. The average SUS score was 81.25, generally considered “Acceptable.” Qualitative themes included RxFill usefulness, ease of use, and fit into current workload. Participants reported RxFill data was useful to estimate patient adherence, however, running reports for each medication was time intensive. Participants had minimal time to review patient charts prior to appointments, which would limit their ability to use RxFill in practice. Conclusions RxFill data, although useful, was not entirely easy to use and did not fit into participants’ current workload. Institutions implementing RxFill should make dispense dates and fill indicators easy to access and consider workload and integration into other health IT systems. Additionally, RxFill use is limited by community pharmacy adoption and the ability to send fill data. Medication adherence Pharmacy Usability Figures Figure 1 Figure 2 Figure 3 BACKGROUND Over 50% of patients are non-adherent to their prescribed medications. ( 1 , 2 ) Medication non-adherence is associated with poor health outcomes and increased risks for hospitalizations and death, resulting in over $ 100 billion of avoidable healthcare costs in the United States annually.( 2 – 4 ) Prescribers, including primary care providers (e.g., Doctors of Medicine [MD] and Osteopathic Medicine [DO] as well as Advanced Practice Providers [APNP, PA] practicing in internal medicine, family medicine, or pediatrics departments), can address non-adherence and promote correct medication use by developing trusting relationships with patients in a blame-free environment, engaging patients and their caregivers in discussions about their disease and medication therapies, and prescribing affordable medications with easy-to-follow directions.( 1 , 2 ) Current methods for obtaining patient medication use and determining adherence in primary care settings are unreliable. The two most common approaches to acquiring medication adherence information are asking patients about their medication-taking behavior and objectively examining pharmacy refill records or insurance claims. Although patient self-reports are the most utilized source of medication adherence for prescribers, ( 5 – 7 ) such data are problematic because patients often overestimate their medication adherence, sometimes by as much as 200%.( 1 , 2 , 8 , 9 ) This adherence overreporting may be unintentional—patients simply do not recall or misremember their medication-taking behavior.( 1 , 2 ) Patients may also intentionally overestimate their medication adherence for numerous reasons, such as embarrassment, provider mistrust, or a social desirability bias to tell the provider what they think they want to hear.( 2 ) Inaccurate patient self-reports are problematic because they may lead to incorrect provider assumptions about the effectiveness of medication therapies for treating conditions. Patient adherence data can also be sent directly to the prescriber from the patient’s insurance or pharmacy benefit manager (PBM). Insurance companies use algorithms to calculate adherence by estimating the number of days in which the patient has access to their medication using the dates the prescriptions were filled at the pharmacy (the date the prescription was billed to the insurance, packaged, labeled, and reviewed) as well as the dates the prescriptions were dispensed (given to the patient).( 10 ) Adherence calculations and data from PBMs, although objective, are problematic and have limitations. First, calculations assume that, just because patients possess a medication, they take it correctly.( 1 , 2 , 4 , 10 ) Patients may, instead, take a medication home from the pharmacy and then forget to take it daily or take it sporadically to save on costs. Second, calculations use dispensing information to estimate patient adherence. This dispensing information is often only available from the community pharmacy that filled the prescription or the patient’s pharmacy insurance or PBM that paid for the prescription. Also, PBMs may periodically send letters or faxes to inform prescribers of patient non-adherence, but these letters arrive months after the patient’s initial non-adherence and may not coincide with patients’ appointments and can be time consuming for prescribers to address. For prescribers to efficiently use objective medication adherence data (including PBM fill data), information must be presented when most clinically beneficial—during the patient’s encounter. RXFILL Health information technology (HIT) has made great strides in facilitating communication between prescribers and pharmacies, including electronic prescribing.( 11 – 13 ) RxFill is an HIT functionality that integrates patients’ prescription fill data from the pharmacy directly into the patients’ record in the clinic’s EHR (illustrated in Fig. 1 ).( 14 ) RxFill is part of the United States National Council for Prescription Drug Programs (NCPDP) prescription transaction (SCRIPT) Implementation Recommendations.( 15 ) Once in the EHR, RxFill displays the prescription dispensing dates, which can be used to infer patient adherence based on refill cadence (illustrated in Fig. 2 ). To locate RxFill data within the EHR, prescribers and clinic team members must run a report for each medication of interest. During a normal patient visit, prescribers require access to large amounts of clinical information in a short period of time. During an encounter, prescribers rapidly retrieve and use EHR information to inform their decision making, such as changing a patient’s medication. How information is presented to prescribers is important to support their decision making. Using and adopting HIT is often hindered by poor design when the HIT is difficult to learn and complicated to use. As such, the way in which RxFill information is designed and implemented into the EHR is crucial to ensure its actual use.( 16 – 19 ) Objective The goal of this study was to examine how primary care prescribers use RxFill during a simulated patient case, identifying how prescribers integrate RxFill into clinical workflow and perceptions of RxFill usability and design. This was accomplished via “think-aloud” usability testing.( 20 ) MATERIALS AND METHODS Setting This study capitalized on RxFill implementation at [Affiliation] in June 2022. [Affiliation], the integrated health system of the [Affiliation], serves more than 700,000 patients each year in the Upper Midwest of the United States at 7 hospitals and 80 outpatient sites. [Affiliation] employs over 1,800 physicians, 190 pharmacists and 21,000 employees. [Affiliation] has 18 dedicated pharmacists that provide ambulatory care services throughout the [Affiliation] primary care clinics. [Affiliation] uses Epic (Epic Systems Inc., Verona, WI) as their EHR vendor.( 21 , 22 ) For RxFill data to flow into the clinic’s EHR, community pharmacies must also “support” and enable RxFill transmission. Around the time of data collection in October 2022, only 206 Wisconsin community pharmacies were able to send RxFill data (pharmacy list provided by Surescripts). These 206 pharmacies consisted of independent or small local chains. No large national chains were included. During the entire project duration, study team member (PK, an internal medicine doctor at [Affiliation]) received only one RxFill transaction. The limited number of pharmacies that have enabled RxFill provides an important context when considering 1) study participants’ knowledge of and familiarity with RxFill and 2) implications for study results and generalizability. For the purposes of this study, we assumed that RxFill data was readily provided and available for all prescriptions, anticipating that study findings would provide momentum for RxFill’s eventual adoption by more community pharmacies. Running an RxFill Report Within [Affiliation], RxFill data were only available as reports during a patient encounter. Encounters included office visits, telephone calls, outpatient labs, or secure patient messaging. To access RxFill data, prescribers were required to: Navigate to patient medication information (via the “Medication” or “Take Action” tab) within the open encounter in the patient’s chart. Click on the desired medication to expand the medication information. Click on the “Report” hyperlink to open the “Order Report.” RxFill is populated in Epic as “Dispense History from External Pharmacies” (Fig. 2 ). Conceptual Framework This study utilized the Unified Theory of Acceptance and Use of Technology (UTAUT) to explain prescribers’ reactions to RxFill and intention to use RxFill in practice (Fig. 3 ).( 23 ) UTAUT was adapted from the Theory of Planned Behavior and places emphasis on the context in which the health IT is implemented (as described above) and other facilitators and barriers that impact its use. UTAUT depicts four main determinants to prescribers’ intention to use RxFill (also called “acceptance”): performance expectancy, effort expectancy, social influence, and facilitating conditions.( 24 ) Performance expectancy describes the degree to which the health IT will enhance or diminish an individual’s job performance. Effort expectancy describes how easy or hard the health IT is for the individual to use. Social influence depicts an individual’s perception of the degree to which other people will approve or disapprove of the health IT use. And, facilitating conditions depict factors that impede or facilitate the behavior, including other work system or contextual influences. Data Collection and Data Evaluation [Affiliation] primary care prescribers were eligible to participate in this study. Individuals were required to be employed and practicing at [Affiliation] at the time of participation. Convenience sampling was used to recruit study participants. As a research team, we worked with the [Affiliation] Interim Chief Medical Information Officer to present the study to potential participants. Recruitment emails were sent using a secure university-issued email account to all General Internal Medicine primary care providers. Prior to data collection, the study was reviewed and approved by the [Affiliation] Institutional Review Board (Submission Identification Number: 2022-0862-CP001; Clinical trial number: not applicable). The study aimed to recruit nine (n = 9) primary care prescribers. The Nielson Norman group recommends conducting qualitative usability studies with five users to maximize the cost-benefit ratio.( 25 ) Nine participants were selected, a priori, to reach saturation of qualitative themes.( 26 ) The study explored how prescribers interacted with RxFill and their perceptions of RxFill usability and design. “Think aloud” usability testing with [Affiliation] primary care prescribers captured commentary that allowed for subsequent qualitative analysis. The methodology was based on Li et. al.’s 2011 “Phase I: usability testing using ‘think aloud’ protocol analysis.”( 20 ) Our study team developed a simulated patient case within [Affiliation]'s EHR test platform (Epic Playground), as well as a semi-structured cognitive interview guide that included instructions to navigate through the simulated case and EHR (interview guide provided in Appendix 1). The case was designed to highlight several medications with various instances of medication adherence. For example, the RxFill data demonstrated the simulated patient filled their 90-day-supply of metformin every 100 to 110 days. In comparison, the dates showed that the patient filled their albuterol inhaler every 15 days, with directions to use only as needed for asthma control. During data collection, the participant followed the scripted navigational instructions to view and review the simulated patients' data, view RxFill data, and assess patient adherence (example shown in Fig. 2 ). Throughout the case, we prompted the participant to “think aloud” and verbalize their thoughts while navigating through each step. Additionally, we probed with additional questions throughout the interview. After completing the case, we asked the participant follow-up questions to elicit general attitudes towards RxFill and its integration into the EHR. We used the UTAUT conceptual framework to inform the creation of the cognitive interview guide probes and follow-up questions (available in Appendix 1). For example, the UTAUT concept “Performance Expectancy” was operationalized as “How useful would [RxFill] be in your typical practice?” (Question 24, Appendix 1). As a second example, UTAUT’s “Effort Expectancy” was phrased as, “How easy do you think it would be to find and use [RxFill]in your typical practice?” (Question 28, Appendix 1). Participants also completed a brief demographic survey and System Usability Scale (SUS) survey. The entire data collection session took approximately 45 minutes and participants received $ 100 remuneration for their time. We observed the participant completing the steps and audio recorded the encounter. The think-aloud and interview audio recording was transcribed verbatim by a professional service. Data Analysis The qualitative thematic analysis was conducted using NVivo 20 (NVivo, Lumivero) following the methodology of Braun and Clarke, using a codebook approach (2019).( 27 – 29 ) Two study team members (TW and AG) read the interview think-aloud commentary and interview transcripts. We developed a codebook (Appendix 2) guided generally by the constructs from the UTAUT conceptual framework (e.g., RxFill Performance Expectancy, Effort Expectancy, and Facilitating Conditions). After the first interview, we expanded the codebook to further explore UTAUT’s “Facilitating Conditions” and “Other Factors” and situate RxFill within the larger context of primary care practice—1) RxFill use was likely dependent on prescribers’ general clinical practice, how they anticipated patient issues, and their comfort with the EHR; as well as 2) how prescribers perceived medication adherence and how they normally engaged patients in conversations surrounding medication adherence. Together, we (TW and AG) used the codebook to synchronously code one transcript and achieve complete agreement. We then independently coded the remaining seven transcripts using deductive analysis.( 30 , 31 ) Transcripts were compared to identify inter-coder reliability (average 98.94% agreement); any discordance was discussed until 100% concordance was reached between the two coders. Coded transcripts were then aggregated into themes according to the UTAUT dimensions of usability. Exemplar quotes were chosen to represent themes in the final manuscript. The SUS survey was scored and analyzed via descriptive statistics in R Statistical Software (v4.3.1; R Core Team 2021).( 32 , 33 ) Generally, scores ranging between 0 to 50 are considered “Not Acceptable,” scores between 51 and 70 are considered “Marginal,” and scores greater than 70 are considered “Acceptable.”( 32 ) RESULTS Recruitment emails were sent to approximately 115 individuals in [Affiliation] department of General International Medicine, with additional convenience sampling during [Affiliation] meetings and seminars. Eight primary care prescribers participated in the study (6% response rate, 1 short of a priori recruitment goal), a majority being white (87.5%) and female (87.5%). On average, participants reported 19.5 years of healthcare experience (ranging from 12 to 29 years) and were 43.5 years old (ranging from 37 to 52 years old). The SUS survey yielded an average score of 81.25, ranging from 65 to 97.5, indicating it was considered “Acceptable.” Qualitative Assessment of Themes The think-aloud commentary and interviews shed light on several themes regarding RxFill acceptance and usability that aligned with UTAUT—Usefulness of RxFill, Ease of Use of RxFill, Facilitating Conditions to Use RxFill, and Intention to Use RxFill. Exemplary quotes are provided for each theme discussed. Theme: Usefulness of RxFill Of note, none of the participants knew about RxFill or the new functionality prior to participating in the study. However, most of the participants stated that the dispense dates provided by RxFill were useful. I think it’s useful for the conversation with the patient, but then also I’m sure it would be helpful for the staff to have as well, so if there’s an issue with how the prescription is written. But, it would guide how I talk to [patients] because it tells me how they’re taking the medications indirectly. – Participant 1 Most participants stated that RxFill and the dispense dates were particularly useful for complex patients, or specific disease states such as diabetes or high cholesterol. I think this would be a point of care if something doesn’t make sense in terms of [the patient’s] A1C or their blood pressure or their lipids, this is where I’d go. But I think albuterol would be a really good [medication] to look at for control data, although I’m very aware that people use each other’s albuterol inhalers in households. And then I think any other PRN for a condition we’re following like chronic pain, it might be useful to check this before I go in the room. – Participant 8 Participants also stated that the dispense dates were useful to compare against patient vitals, lab results, therapy outcomes, or other patient reports. I think where I might use it more is not necessarily preparing [for the visit] but after I’ve talked to [the patient], or if we’re seeing [their] cholesterol is not controlled, then I can go back and ask, “are they actually, getting their medication?” Or if as I am preparing, I see that their cholesterol isn’t at goal, then this might be something would pull up and dig into. – Participant 4 Theme: Ease of Use of RxFill While the participants agreed that RxFill data and the dispense information was useful and easy to comprehend to approximate adherence, their responses varied on how easy RxFill was to locate and access, citing difficulties navigating the EHR to find the data. For some participants who previously had their medical assistants (MAs) and clinic staff call patients’ pharmacies to obtain adherence information, RxFill was an easy, time-saving functionality. I think it would be really helpful. I didn’t realize I could see when these pharmacies are filling things. And that used to always take me asking my MA to call the pharmacy, and they’re always busy, so I don’t do that very often unless it’s more critical like a new patient, we don’t know exactly what dose they’re on and things. So I think it would be incredibly helpful. – Participant 2 Conversely, other participants reported that RxFill, as implemented, was not easy to use and that it was cumbersome to run reports for each medication of interest. [RxFill’s] useful, but it’s fairly onerous because, you would have to click on the report button and look through [each medication] every time. And to look through every medication would take you several minutes at least, and so I would probably only use it in select cases. – Participant 3 The way [RxFill] sits, it’s not practical at all because it takes several clicks, it takes a couple clicks to get to. And I have really small report link, and when I see a report link, it usually just means I get some sort of report that has a bunch of stuff in it. – Participant 1 Theme: Facilitating Conditions/Personal and Work System Characteristics Several work system and personal characteristics influenced RxFill usability. Most participants stated that their busy workload prohibited them from spending too much time on reviewing the medication list before the encounter, including the new RxFill functionality. One participant stated that, in general, they’re often running late and have less time to prep. If I’m running late, I do less prep, which is arguably a terrible idea. […] But honestly, I’m usually not looking at meds before I go in the room. My medical assistant does a [medication] review, and then, when I’m going over the medical issues list in the room with the patient, then that’s when I’ll ask questions about medication adherence and tolerance. – Participant 8 I don’t think I would look at every medication for every patient for sure. Would not have time to do that. So, I likely wouldn’t do it at every encounter, any, every visit, but it’s very easy to find and available. – Participant 2 Theme: Intentions to Use RxFill RxFill usefulness, ease of use, other facilitating conditions, and trust in the data, combined to indicate the participants’ intentions to use RxFill in their actual practice. Most participants indicated that they could see themselves using RxFill and the filling data in “some” capacity, but only in certain scenarios. For example, participants reported that they could use RxFill data as a resource when triggered by another source, such as an out-of-range vital, lab, or patient report. Others mentioned that the usefulness of RxFill data was limited on its own, several even questioning the reliability of RxFill dispense date, but that the information may be helpful to create a holistic picture of a patients’ medication experience. I guess I would want to know how reliable [RxFill] is. Oftentimes, when you look at things like [the Prescription Drug Monitoring Program] for refills, sometimes it’s not accurate. So, I want to know how accurate is this? Does every pharmacy report it? Or maybe they got it at a pharmacy out of state, does it not report it then? So what pharmacies are on this? How reliable is it? – Participant 2 I feel like I’m not in a place where I trust that refill data is the only piece of data I need on adherence. This is more information than I knew I had here, and now I know how to access it. […] I think that now that I know it exists, I’ll actually access this. But I just don’t have time to do this for every single patient. – Participant 8 Other participants stated they may look at RxFill when “in the room” with the patient in real-time but may not use RxFill to prepare for an encounter. If someone was coming in for an annual review, when there’s so many things to go over, I probably wouldn’t [look at RxFill] beforehand. If someone was coming specifically for asthma, I might look at [RxFill] beforehand. Whereas if I’m planning an encounter, and I could look at it right away, it could be a little more real time and able to address it in real time. – Participant 4 DISCUSSION These findings suggested overall support for RxFill, as implemented, being potentially useful and helpful to engage patients in conversations around medication adherence. Ultimately, however, when coupled with the limited inflow of data from community pharmacies (described in the Setting section), RxFill was not likely to be actually used in practice. Issue 1: RxFill’s Usefulness in Real-Time During Patient Encounters Participant Feedback When originally proposed, we hypothesized RxFill would be most useful when anticipating patient concerns before an encounter, or when doing a chart review prior to a visit—prescribers would be actively searching for and evaluating non-adherence before walking into the patient room.( 1 , 5 – 7 , 34 ) Participant feedback, however, revealed that many prescribers do not have time for such an in-depth analysis within their already busy workload, a sentiment echoed in the literature.( 35 – 37 ) Additionally, one participant reported that they do not consider medication adherence at all prior to a patient encounter, instead focusing on other outcomes such as lab results, vitals, or symptoms. Once in the room with the patient, they relied on patient reports and were hesitant to utilize objective metrics of medication use such as dispense dates alone. Participants also questioned the reliability of RxFill data, based on their prior experiences with inadequate reporting from other HIT functionalities.( 10 , 17 , 38 , 39 ) Recommendations As designed, RxFill use is dependent upon the value prescribers ascribe to evaluating medication non-adherence.( 1 ) RxFill requires prescribers to actively retrieve data, as opposed to other functionalities that flag or alert non-adherent behaviors.( 40 , 41 ) In considering best practices and current design, RxFill may be most useful when in the room with the patient as a way to quickly and objectively assess medication adherence in real-time. Prescribers should consider RxFill as another source of information when patient outcomes or reports do not align with medication therapy. Prescribers who leverage RxFill data should use their clinical and motivational interviewing skills, such as using patient-centered language and non-judgmental and non-biased tones, to engage patients in conversations around medication adherence.( 42 ) Participants also mentioned the utility of RxFill for other clinic staff members. For example, having RxFill information may be helpful for Medical Assistants (MA) or other clinic personnel while reviewing medication lists with patients at the beginning of an encounter.( 43 ) Similarly, RxFill information may be helpful for nurses when answering patient or pharmacist questions regarding prescriptions.( 44 , 45 ) Additionally, with pharmacists now often embedded into primary care or other ambulatory practices, having access to RxFill information may be helpful when conducting Comprehensive Medication Reviews (CMRs) or other medication management services.( 46 , 47 ) Issue 2: RxFill Needs to be Easy and Intuitive to Access and Use Participant Feedback Participants emphasized that RxFill data needs to be easy to access and intuitive to interpret and use. However, most participants reported that they were unlikely to use RxFill in actual practice because it required numerous clicks and was an onerous process, especially for patients with many medications. When we began this study, we hypothesized that RxFill data and adherence assessments would be universally valuable and would be a standard of care for all medications. Practically however, given the high click burden participants stated they more likely to target RxFill use for patients with diabetes or hypertension, where medication adherence is crucial for patient outcomes. Recommendations The EHR contains massive amounts of clinical information; prescribers must rapidly retrieve and use this information to make clinical decisions during a short patient encounter.( 48 ) However, just because information exists within the EHR does not mean it will be used as intended.( 16 – 19 , 48 ) A study of drug safety alerts found that the alerts were overridden up to 96% of the time.( 49 ) As evident by our findings, even a new IT that is deemed “simple,” such as RxFill, may not be used if providers 1) do not know that the information exists, 2) cannot find the information readily in the EHR, or 3) cannot quickly interpret and use the information during a patient visit.( 18 , 20 , 50 – 52 ) When organizations are choosing to implement RxFill, they should consider a design that is intuitive and does not take any additional time beyond prescriber current practice. The new technology should be time “neutral” (meaning it does not add any additional time) or time “negative” (meaning it reduces the time necessary to complete a task).( 24 , 53 , 54 ) Organizations should utilize end-user input to determine how prescribers currently access medication information to tailor RxFill to best suit their prescribers’ needs.( 18 , 55 ) For example, where do prescribers access and review patient medications? This may provide insight on where to integrate RxFill data. Should medical assistants or nurses be responsible for reviewing RxFill information and triaging complex patients prior to the encounter? This may be helpful to standardize practice protocols or distribute tasks amongst team members. Additionally, organizations and prescribers should consider prioritizing RxFill use and assessing medication adherence for “high risk” medications or disease states.( 40 , 56 ) While adherence is still critical to evaluate for patients, we recognize that time, workload, and other contextual restraints make this standard difficult to implement in practice. Therefore, it’s important to make RxFill use as easy as possible to promote its actual use. Issue 3: RxFill Use is Limited by Community Pharmacy Adoption Participant Feedback Ultimately prescribers saw the value and usefulness of RxFill, even when its actual use was somewhat undermined by its design. A major limitation to RxFill’s actual use was the lack of adoption by community pharmacies and flow of information between community pharmacies and clinic EHR. Recommendations While we recommend that community pharmacies support sending RxFill data and transactions, we recognize that barriers, such as Surescripts transaction costs, may limit implementation in practice. For RxFill to be adopted by community pharmacies, and therefore adopted by healthcare systems, organizations, and prescribers, future work should attempt to reduce barriers that limit its use. Strengths and Limitations As evident by these potential RxFill issues and recommendations, the central limitation of this study is the lack of broad generalizability. This study took place in one midwestern academic health system and researchers had organizational support and buy-in throughout RxFill implementation. Additionally, the usability study and qualitative interviews were limited to the perspectives of those who chose to participate, indicating the potential for selection bias (self-selection of prescribers who may have higher comfort using the EHR or interest in medication adherence). Amidst these limitations, however, the study leveraged the UTAUT framework and “think aloud” protocols to examine usability of RxFill. The study provided recommendations for organizations intending to implement RxFill to ensure health IT success. CONCLUSIONS This project demonstrated the potential value of RxFill—providing the right information, to the right person, at the right time. RxFill provided the “right information” by communicating accurate dispensing and medication fill information that can be utilized to inform patient adherence calculations, clinical decisions, and conversations with patients. RxFill provided this information to the “right person” by sharing information from the patient’s pharmacy to providers. Finally, RxFill provided this information at the “right time”—within the EHR at the time of encounter before and during the appointment. Specifically, by delivering patient fill and adherence data at the right time, RxFill potentially facilitates conversations between patients and providers that can improve medication adherence and resulting clinical outcomes. However, this project also demonstrated key barriers in RxFill design and infrastructure that limited prescribers’ intentions to use in actual practice. While we provide recommendations for other health systems, clinics, and prescribers planning to utilize RxFill, this study also highlights the importance of usability evaluation as new HIT functionalities are trialed and implemented into the standard of care. Abbreviations APNP Advanced practice Nurse Practitioner EHR Electronic Health Record DO Doctor Osteopathic Medicine HIT Health Information Technology MD Doctor of Medicine [MD] NCPDP National Council for Prescription Drug Programs PA Physician Assistant SUS System Usability Scale UTAUT Unified Theory of Acceptance and Use of Technology Declarations Ethics approval and consent to participate: Ethics approval was obtained before data collection with a waiver of signed consent. This study was approved by the Health Sciences Institutional Review Board (HS IRB) at the University of Wisconsin-Madison(ID: 2023-0498). The HS IRB reviews and approves research in accordance with the laws of the United States of America and the State of Wisconsin. The IRB complies with the applicable requirements of the Department of Health and Human Services (DHHS) regulations, 45 CFR Part 46; the Food and Drug Administration (FDA) regulations, 21 CFR Parts 50, 56, 312, and 812; Veteran’s Administration (VA) Regulations pertaining to the protection of human subjects, 38 CFR Part 16; and the privacy requirements of the Health Insurance Portability and Accountability Act of 1996 implemented by 45 CFR Parts 160 and 164 (Privacy Rule). Clinical Trial Registration: Not applicable Consent for publication: Participant information sheet (waiver of signed consent) included permission to publish de-identified statements. Availability of data and materials: The datasets generated and/or analyzed during the current study are not publicly available to protect the identity of the participants but are available from the corresponding author on reasonable request and approval from UW Health. Competing Interests: The authors declare that they have no competing interests. Funding: This project was supported by the National Council for Prescription Drug Programs (NCPDP) Foundation. TW was supported by the University of Wisconsin Primary Care Research Fellowship, funded by grant T32HP10010 from the Health Resources and Services Administration. Authors’ Contributions: TW was responsible for conceptualization, methodology, analysis, writing – original draft, project administration, and funding acquisition. AG was responsible for conceptualization, methodology, analysis, writing – review & editing. JS and PK were responsible for conceptualization, methodology, writing – review & editing. Acknowledgements: Chelsea Steitz and Emily Hoffins References Brown MT, Bussell JK. Medication adherence: WHO cares? Mayo Clin Proc. 2011;86(4):304–14. Brown MT, Bussell J, Dutta S, Davis K, Strong S, Mathew S. Medication Adherence: Truth and Consequences. Am J Med Sci. 2016;351(4):387–99. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487–97. Heidenreich PA. Patient adherence: the next frontier in quality improvement. Am J Med Elsevier BV. 2004;117(2):130–2. Kwan YH, Weng SD, Loh DHF, Phang JK, Oo LJY, Blalock DV, et al. Measurement Properties of Existing Patient-Reported Outcome Measures on Medication Adherence: Systematic Review. J Med Internet Res. 2020;22(10):e19179. Nguyen T-M-U, La Caze A, Cottrell N. What are validated self-report adherence scales really measuring? a systematic review. Br J Clin Pharmacol. 2014;77(3):427–45. Schönfeld MS, Pfisterer-Heise S, Bergelt C. Self-reported health literacy and medication adherence in older adults: a systematic review. BMJ Open. 2021;11(12):e056307. Hebel S, Kahn-Woods E, Malone-Thomas S, McNeese M, Thornton L, Sukhija-Cohen A, et al. Brief Report: Discrepancies Between Self-Reported Adherence and a Biomarker of Adherence in Real-World Settings. J Acquir Immune Defic Syndr. 2020;85(4):454–7. Krueger KP, Berger BA, Felkey B. Medication adherence and persistence: a comprehensive review. Adv Ther. 2005;22(4):313–56. Prieto-Merino D, Mulick A, Armstrong C, Hoult H, Fawcett S, Eliasson L, et al. Estimating proportion of days covered (PDC) using real-world online medicine suppliers’ datasets. J Pharm Policy Pract. 2021;14(1):113. Watterson TL, Stone JA, Gilson A, Brown R, Xiong KZ, Schiefelbein A et al. Impact of CancelRx on discontinuation of controlled substance prescriptions [Internet]. bioRxiv. medRxiv; 2021. Available from: http://medrxiv.org/lookup/doi/ 10.1101/2021.01.12.21249700 Watterson TL, Stone JA, Brown R, Xiong KZ, Schiefelbein A, Ramly E et al. CancelRx: a health IT tool to reduce medication discrepancies in the outpatient setting. J Am Med Inform Assoc [Internet]. 2021; Available from: http://dx.doi.org/10.1093/jamia/ocab038 Rupp MT, Warholak TL. Evaluation of e-prescribing in chain community pharmacy: best-practice recommendations. J Am Pharm Assoc. 2008;48(3):364–70. National Council for Prescription Drug Programs. NCPDP Electronic Prescribing Standards [Internet]. 2014. Available from: https://www.ncpdp.org/NCPDP/media/pdf/NCPDPEprescribingBasics.pdf SCRIPT Implementation Recommendations [Internet]. National Council for Prescription Drug Programs. 2023 May. Available from: https://www.ncpdp.org/NCPDP/media/pdf/SCRIPT-Implementation-Recommendations.pdf Harrison MI, Koppel R, Bar-Lev S. Unintended Consequences of Information Technologies in Health Care–An Interactive Sociotechnical Analysis [Internet]. Vol. 14, Journal of the American Medical Informatics Association. 2007. pp. 542–9. Available from: http://dx.doi.org/10.1197/jamia.m2384 Karsh B-T, Weinger MB, Abbott PA, Wears RL. Health information technology: fallacies and sober realities. J Am Med Inf Assoc. 2010;17(6):617–23. Karsh B-T. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care. 2004;13(5):388–94. Karsh B-T, Holden RJ, Alper SJ, Or CKL. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care. 2006;15(Suppl 1):i59–65. Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, et al. Integrating usability testing and think-aloud protocol analysis with near-live clinical simulations in evaluating clinical decision support. Int J Med Inf. 2012;81(11):761–72. Health. Information Exchange (HEI) [Internet]. [cited 2024 Feb 8]. Available from: https://www.uwhealth.org/health-information-exchange-hie Merchant P. The epic. Verona, WI: Routledge; 2017. Venkatesh V, Morris MG, Davis GB, Davis FD. User Acceptance of Information Technology: Toward a Unified View. Miss Q. 2003;27(3):425–78. Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inf. 2010;43(1):159–72. Nielsen J. How Many Test Users in a Usability Study? [Internet]. Nielsen Norman Group. 2012 [cited 2025 Jul 3]. Available from: https://www.nngroup.com/articles/how-many-test-users/ Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual Res Sport Exerc Health. 2021;13(2):201–16. QSR International (Lumivero). NVivo 12. 2017. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. Braun V, Clarke V, Hayfield N, Terry G. Thematic Analysis. Handbook of Research Methods in Health Social Sciences. Singapore: Springer Singapore; 2019. pp. 843–60. Crotty M, Crotty MF. The Foundations of Social Research: Meaning and Perspective in the Research Process. SAGE; 1998. p. 248. Rezigalla AA. Observational Study Designs: Synopsis for Selecting an Appropriate Study Design. Cureus. 2020;12(1):e6692. Brooke J. SUS: A quick and dirty usability scale. Usability Eval Ind. 1995;189. R Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing. 2021. Available from: https://www.R-project.org/ Basu S. Accurately estimating medication non-adherence through patient self-report: possibilities and limitation of a new scale. Curr Med Res Opin. 2021;37(8):1349–51. Lee M, Leonard C, Greene P, Kenney R, Whittington MD, Kirsh S, et al. Perspectives of VA primary care clinicians toward electronic consultation-related workload burden: A qualitative analysis: A qualitative analysis. JAMA Netw Open. 2020;3(10):e2018104. Rittenberg E, Liebman JB, Rexrode KM. Primary care physician gender and electronic health record workload. J Gen Intern Med. 2022;37(13):3295–301. Agarwal SD, Pabo E, Rozenblum R, Sherritt KM. Professional dissonance and burnout in primary care: A qualitative study: A qualitative study. JAMA Intern Med. 2020;180(3):395–401. Tung F-C, Chang S-C, Chou C-M. An extension of trust and TAM model with IDT in the adoption of the electronic logistics information system in HIS in the medical industry. Int J Med Inf. 2008;77(5):324–35. Luo J, Wong R, Mehta T, Schwartz JI, Epstein JA, Smith E, et al. Implementing real-time prescription benefit tools: Early experiences from 5 academic medical centers. Healthc (Amst). 2023;11(2):100689. Zaugg V, Korb-Savoldelli V, Durieux P, Sabatier B. Providing physicians with feedback on medication adherence for people with chronic diseases taking long-term medication. Cochrane Database Syst Rev. 2018;1(1):CD012042. Kane-Gill SL, O’Connor MF, Rothschild JM, Selby NM, McLean B, Bonafide CP, et al. Technologic distractions (part 1): Summary of approaches to manage alert quantity with intent to reduce alert fatigue and suggestions for alert fatigue metrics. Crit Care Med. 2017;45(9):1481–8. McQuaid EL, Landier W. Cultural Issues in Medication Adherence: Disparities and Directions. J Gen Intern Med. 2018;33(2):200–6. Reedy AB, Yeh JY, Nowacki AS, Hickner J. Patient, physician, medical assistant, and office visit factors associated with medication list agreement. J Patient Saf. 2016;12(1):18–24. Norful A, Martsolf G, de Jacq K, Poghosyan L. Utilization of registered nurses in primary care teams: A systematic review. Int J Nurs Stud. 2017;74:15–23. Celio J, Ninane F, Bugnon O, Schneider MP. Pharmacist-nurse collaborations in medication adherence-enhancing interventions: A review. Patient Educ Couns. 2018;101(7):1175–92. Yoo A, Fennelly JE, Renauer MM, Coe AB, Choe HM, Marshall VD et al. Comprehensive medication review service by embedded pharmacists in primary care: Innovations and impact. J Am Pharm Assoc (2003). 2022;62(2):580–587.e1. Gernant SA, Bacci JL, Upton C, Ferreri SP, McGrath S, Chui MA, et al. Three opportunities for standardization: A literature review of the variation among pharmacists’ patient care services terminology. Res Social Adm Pharm. 2020;16(6):766–75. Gawande A. Why Doctors Hate Their Computers. The New Yorker [Internet]. 2018; Available from: https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers Karsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. Rockville, MD: Agency for Healthcare Research and Quality [Internet]. 2009;200943. Available from: https://healthit.ahrq.gov/sites/default/files/docs/biblio/09-0054-EF-Updated_0.pdf Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inf. 2009;78(5):340–53. Ratwani RM, Savage E, Will A, Arnold R, Khairat S, Miller K, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inf Assoc. 2018;25(9):1197–201. Brunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inf. 2017;104:56–64. Karsh B-T, Escoto KH, Beasley JW, Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon. 2006;37(3):283–95. Gardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inf Assoc. 2019;26(2):106–14. Woodward M, Dixon-Woods M, Randall W, Walker C, Hughes C, Blackwell S et al. How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study. BMJ Qual Saf [Internet]. 2023 Dec 12 [cited 2023 Dec 18]; Available from: https://qualitysafety.bmj.com/content/early/2023/12/12/bmjqs-2023-016196?utm_campaign=Research%20papers&utm_medium=email&_hsmi=286361573&_hsenc=p2ANqtz-9JhxxIS9NwPR7FWomvykN6DPuDul4XbMyl_PVoj61UI_krOXq7EiYzbFL0lSn_DfrXv8FAdNXYKLA53_A8xIf3-aMIhQ&utm_content=286361573&utm_source=hs_email Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inf. 2020;11(1):46–58. Additional Declarations No competing interests reported. Supplementary Files Appendix1.InterviewGuide.docx Appendix2.Codebook.docx Cite Share Download PDF Status: Published Journal Publication published 16 Apr, 2026 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted Editorial decision: Revision requested 24 Mar, 2026 Reviews received at journal 25 Aug, 2025 Reviewers agreed at journal 19 Aug, 2025 Reviews received at journal 15 Aug, 2025 Reviewers agreed at journal 15 Aug, 2025 Reviewers invited by journal 15 Aug, 2025 Submission checks completed at journal 10 Jul, 2025 First submitted to journal 07 Jul, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5485583","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":501151351,"identity":"164d8683-18cf-4559-9d44-892866fd6556","order_by":0,"name":"Taylor L. Watterson","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAoklEQVRIiWNgGAWjYDACdjBpw8AGoniI0cHDDCQOJKSRruUwlEeMFntmHsPPH3+cT+xjP8D44G0bUbbwGEscSLid2MaTwGw4lzgtvBsgWiQY2KR5idSy+ceBhHMgLey/idWyDWjLAbAtzMRpOcz/zeJMWrJxG09is+Scc0RoYW9vS75RYWMnO7/98MEPb8qI0IIEGBtIUz8KRsEoGAWjADcAAPHTMCKmnMmqAAAAAElFTkSuQmCC","orcid":"","institution":"University of Illinois Chicago Retzky College of Pharmacy","correspondingAuthor":true,"prefix":"","firstName":"Taylor","middleName":"L.","lastName":"Watterson","suffix":""},{"id":501151352,"identity":"9ba679bc-3f27-4cb3-a684-2796fd4b3021","order_by":1,"name":"Aaron M. Gilson","email":"","orcid":"","institution":"University of Wisconsin-Madison School of Pharmacy","correspondingAuthor":false,"prefix":"","firstName":"Aaron","middleName":"M.","lastName":"Gilson","suffix":""},{"id":501151353,"identity":"267c9de2-2097-4500-a458-de1df6cd353c","order_by":2,"name":"Peter C. Kleinschmidt","email":"","orcid":"","institution":"University of Wisconsin School of Medicine and Public Health","correspondingAuthor":false,"prefix":"","firstName":"Peter","middleName":"C.","lastName":"Kleinschmidt","suffix":""},{"id":501151354,"identity":"d25c8e9a-6f34-4157-9783-160ab2e54dcb","order_by":3,"name":"Jamie A. Stone","email":"","orcid":"","institution":"University of Wisconsin-Madison School of Pharmacy","correspondingAuthor":false,"prefix":"","firstName":"Jamie","middleName":"A.","lastName":"Stone","suffix":""}],"badges":[],"createdAt":"2024-11-19 17:53:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5485583/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5485583/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12911-026-03496-z","type":"published","date":"2026-04-16T15:58:56+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":89271717,"identity":"b2b8712e-b943-475a-bb0d-f75b918abd2e","added_by":"auto","created_at":"2025-08-18 09:01:50","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":48531,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePrescribers’ Source of Medication Adherence Data via RxFill:\u003c/strong\u003e Grey arrows depict flow of medication adherence data through RxFill. Accurate prescription fill and dispensing information for e-prescribed prescriptions flows from the pharmacy dispensing system, through the third-party vendor, Surescripts, into the clinic EHR. This information can be viewed and used by the prescriber and clinic team at the time of the encounter.\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/b03b076ff62b45b5e0668842.jpg"},{"id":89271725,"identity":"57df714f-c7dc-433e-b01d-ebb73c692e58","added_by":"auto","created_at":"2025-08-18 09:01:50","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":96860,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eRxFill Generated Dispense Data\u003c/strong\u003e: Sample medication dispense report and adherence data, generated from Epic Playground (Epic Systems Inc., Verona, WI).\u003c/p\u003e","description":"","filename":"Figure2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/4e751a05116c8aa09e2324fb.jpg"},{"id":89271723,"identity":"1d4a1893-21e1-40f4-814b-faafee24c356","added_by":"auto","created_at":"2025-08-18 09:01:50","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":66238,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eUnified Theory of Acceptance and Use of Technology (UTAUT)\u003c/strong\u003e: A unified model includes four core determinants of technology intention and usage as well as four moderators of key relationships.\u003c/p\u003e","description":"","filename":"Figure3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/b95d1d2dc0c8656b92e55360.jpg"},{"id":107352371,"identity":"d85fdb3e-d750-4568-a3f4-d193c30c8fd9","added_by":"auto","created_at":"2026-04-20 16:13:59","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":656701,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/584a1f6a-34bf-45bc-a2e3-a35e8ae86d18.pdf"},{"id":89271716,"identity":"66be125f-7417-4708-b786-d8e572ba4271","added_by":"auto","created_at":"2025-08-18 09:01:49","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":20736,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix1.InterviewGuide.docx","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/a857f7292c164af4859fe984.docx"},{"id":89273530,"identity":"428e3f86-3396-48cd-9816-6a29fd73d2d6","added_by":"auto","created_at":"2025-08-18 09:09:50","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":16150,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix2.Codebook.docx","url":"https://assets-eu.researchsquare.com/files/rs-5485583/v1/0f1c8e2988bc740d67b4167f.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Examining Usability of RxFill: Integrating Health IT to Support Medication Adherence","fulltext":[{"header":"BACKGROUND","content":"\u003cp\u003eOver 50% of patients are non-adherent to their prescribed medications. (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) Medication non-adherence is associated with poor health outcomes and increased risks for hospitalizations and death, resulting in over \u003cspan\u003e$\u003c/span\u003e100\u0026nbsp;billion of avoidable healthcare costs in the United States annually.(\u003cspan additionalcitationids=\"CR3\" citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e) Prescribers, including primary care providers (e.g., Doctors of Medicine [MD] and Osteopathic Medicine [DO] as well as Advanced Practice Providers [APNP, PA] practicing in internal medicine, family medicine, or pediatrics departments), can address non-adherence and promote correct medication use by developing trusting relationships with patients in a blame-free environment, engaging patients and their caregivers in discussions about their disease and medication therapies, and prescribing affordable medications with easy-to-follow directions.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eCurrent methods for obtaining patient medication use and determining adherence in primary care settings are unreliable. The two most common approaches to acquiring medication adherence information are asking patients about their medication-taking behavior and objectively examining pharmacy refill records or insurance claims. Although patient self-reports are the most utilized source of medication adherence for prescribers, (\u003cspan additionalcitationids=\"CR6\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e) such data are problematic because patients often overestimate their medication adherence, sometimes by as much as 200%.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e) This adherence overreporting may be unintentional\u0026mdash;patients simply do not recall or misremember their medication-taking behavior.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) Patients may also intentionally overestimate their medication adherence for numerous reasons, such as embarrassment, provider mistrust, or a social desirability bias to tell the provider what they think they want to hear.(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) Inaccurate patient self-reports are problematic because they may lead to incorrect provider assumptions about the effectiveness of medication therapies for treating conditions.\u003c/p\u003e\u003cp\u003ePatient adherence data can also be sent directly to the prescriber from the patient\u0026rsquo;s insurance or pharmacy benefit manager (PBM). Insurance companies use algorithms to calculate adherence by estimating the number of days in which the patient has access to their medication using the dates the prescriptions were filled at the pharmacy (the date the prescription was billed to the insurance, packaged, labeled, and reviewed) as well as the dates the prescriptions were dispensed (given to the patient).(\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e) Adherence calculations and data from PBMs, although objective, are problematic and have limitations. First, calculations assume that, just because patients possess a medication, they take it correctly.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e) Patients may, instead, take a medication home from the pharmacy and then forget to take it daily or take it sporadically to save on costs. Second, calculations use dispensing information to estimate patient adherence. This dispensing information is often only available from the community pharmacy that filled the prescription or the patient\u0026rsquo;s pharmacy insurance or PBM that paid for the prescription. Also, PBMs may periodically send letters or faxes to inform prescribers of patient non-adherence, but these letters arrive months after the patient\u0026rsquo;s initial non-adherence and may not coincide with patients\u0026rsquo; appointments and can be time consuming for prescribers to address.\u003c/p\u003e\u003cp\u003eFor prescribers to efficiently use objective medication adherence data (including PBM fill data), information must be presented when most clinically beneficial\u0026mdash;during the patient\u0026rsquo;s encounter.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRXFILL\u003c/b\u003e\u003c/p\u003e\u003cp\u003eHealth information technology (HIT) has made great strides in facilitating communication between prescribers and pharmacies, including electronic prescribing.(\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) RxFill is an HIT functionality that integrates patients\u0026rsquo; prescription fill data from the pharmacy directly into the patients\u0026rsquo; record in the clinic\u0026rsquo;s EHR (illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).(\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e) RxFill is part of the United States National Council for Prescription Drug Programs (NCPDP) prescription transaction (SCRIPT) Implementation Recommendations.(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e) Once in the EHR, RxFill displays the prescription dispensing dates, which can be used to infer patient adherence based on refill cadence (illustrated in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). To locate RxFill data within the EHR, prescribers and clinic team members must run a report for each medication of interest.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eDuring a normal patient visit, prescribers require access to large amounts of clinical information in a short period of time. During an encounter, prescribers rapidly retrieve and use EHR information to inform their decision making, such as changing a patient\u0026rsquo;s medication. How information is presented to prescribers is important to support their decision making. Using and adopting HIT is often hindered by poor design when the HIT is difficult to learn and complicated to use. As such, the way in which RxFill information is designed and implemented into the EHR is crucial to ensure its actual use.(\u003cspan additionalcitationids=\"CR17 CR18\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e)\u003c/p\u003e\u003cp\u003e\u003cb\u003eObjective\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe goal of this study was to examine how primary care prescribers use RxFill during a simulated patient case, identifying how prescribers integrate RxFill into clinical workflow and perceptions of RxFill usability and design. This was accomplished via \u0026ldquo;think-aloud\u0026rdquo; usability testing.(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e)\u003c/p\u003e"},{"header":"MATERIALS AND METHODS","content":"\u003cp\u003e\u003cb\u003eSetting\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis study capitalized on RxFill implementation at [Affiliation] in June 2022. [Affiliation], the integrated health system of the [Affiliation], serves more than 700,000 patients each year in the Upper Midwest of the United States at 7 hospitals and 80 outpatient sites. [Affiliation] employs over 1,800 physicians, 190 pharmacists and 21,000 employees. [Affiliation] has 18 dedicated pharmacists that provide ambulatory care services throughout the [Affiliation] primary care clinics. [Affiliation] uses Epic (Epic Systems Inc., Verona, WI) as their EHR vendor.(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eFor RxFill data to flow into the clinic\u0026rsquo;s EHR, community pharmacies must also \u0026ldquo;support\u0026rdquo; and enable RxFill transmission. Around the time of data collection in October 2022, only 206 Wisconsin community pharmacies were able to send RxFill data (pharmacy list provided by Surescripts). These 206 pharmacies consisted of independent or small local chains. No large national chains were included. During the entire project duration, study team member (PK, an internal medicine doctor at [Affiliation]) received only one RxFill transaction. The limited number of pharmacies that have enabled RxFill provides an important context when considering 1) study participants\u0026rsquo; knowledge of and familiarity with RxFill and 2) implications for study results and generalizability.\u003c/p\u003e\u003cp\u003eFor the purposes of this study, we assumed that RxFill data was readily provided and available for all prescriptions, anticipating that study findings would provide momentum for RxFill\u0026rsquo;s eventual adoption by more community pharmacies.\u003c/p\u003e\u003cp\u003e\u003cb\u003eRunning an RxFill Report\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWithin [Affiliation], RxFill data were only available as reports during a patient encounter. Encounters included office visits, telephone calls, outpatient labs, or secure patient messaging. To access RxFill data, prescribers were required to:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eNavigate to patient medication information (via the \u0026ldquo;Medication\u0026rdquo; or \u0026ldquo;Take Action\u0026rdquo; tab) within the open encounter in the patient\u0026rsquo;s chart.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eClick on the desired medication to expand the medication information.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eClick on the \u0026ldquo;Report\u0026rdquo; hyperlink to open the \u0026ldquo;Order Report.\u0026rdquo; RxFill is populated in Epic as \u0026ldquo;Dispense History from External Pharmacies\u0026rdquo; (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eConceptual Framework\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThis study utilized the Unified Theory of Acceptance and Use of Technology (UTAUT) to explain prescribers\u0026rsquo; reactions to RxFill and intention to use RxFill in practice (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) UTAUT was adapted from the Theory of Planned Behavior and places emphasis on the context in which the health IT is implemented (as described above) and other facilitators and barriers that impact its use. UTAUT depicts four main determinants to prescribers\u0026rsquo; intention to use RxFill (also called \u0026ldquo;acceptance\u0026rdquo;): performance expectancy, effort expectancy, social influence, and facilitating conditions.(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e) Performance expectancy describes the degree to which the health IT will enhance or diminish an individual\u0026rsquo;s job performance. Effort expectancy describes how easy or hard the health IT is for the individual to use. Social influence depicts an individual\u0026rsquo;s perception of the degree to which other people will approve or disapprove of the health IT use. And, facilitating conditions depict factors that impede or facilitate the behavior, including other work system or contextual influences.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cb\u003eData Collection and Data Evaluation\u003c/b\u003e\u003c/p\u003e\u003cp\u003e[Affiliation] primary care prescribers were eligible to participate in this study. Individuals were required to be employed and practicing at [Affiliation] at the time of participation. Convenience sampling was used to recruit study participants. As a research team, we worked with the [Affiliation] Interim Chief Medical Information Officer to present the study to potential participants. Recruitment emails were sent using a secure university-issued email account to all General Internal Medicine primary care providers. Prior to data collection, the study was reviewed and approved by the [Affiliation] Institutional Review Board (Submission Identification Number: 2022-0862-CP001; Clinical trial number: not applicable). The study aimed to recruit nine (n\u0026thinsp;=\u0026thinsp;9) primary care prescribers. The Nielson Norman group recommends conducting qualitative usability studies with five users to maximize the cost-benefit ratio.(\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e) Nine participants were selected, a priori, to reach saturation of qualitative themes.(\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eThe study explored how prescribers interacted with RxFill and their perceptions of RxFill usability and design. \u0026ldquo;Think aloud\u0026rdquo; usability testing with [Affiliation] primary care prescribers captured commentary that allowed for subsequent qualitative analysis. The methodology was based on Li et. al.\u0026rsquo;s 2011 \u0026ldquo;Phase I: usability testing using \u0026lsquo;think aloud\u0026rsquo; protocol analysis.\u0026rdquo;(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eOur study team developed a simulated patient case within [Affiliation]'s EHR test platform (Epic Playground), as well as a semi-structured cognitive interview guide that included instructions to navigate through the simulated case and EHR (interview guide provided in Appendix 1). The case was designed to highlight several medications with various instances of medication adherence. For example, the RxFill data demonstrated the simulated patient filled their 90-day-supply of metformin every 100 to 110 days. In comparison, the dates showed that the patient filled their albuterol inhaler every 15 days, with directions to use only as needed for asthma control.\u003c/p\u003e\u003cp\u003eDuring data collection, the participant followed the scripted navigational instructions to view and review the simulated patients' data, view RxFill data, and assess patient adherence (example shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Throughout the case, we prompted the participant to \u0026ldquo;think aloud\u0026rdquo; and verbalize their thoughts while navigating through each step. Additionally, we probed with additional questions throughout the interview. After completing the case, we asked the participant follow-up questions to elicit general attitudes towards RxFill and its integration into the EHR.\u003c/p\u003e\u003cp\u003eWe used the UTAUT conceptual framework to inform the creation of the cognitive interview guide probes and follow-up questions (available in Appendix 1). For example, the UTAUT concept \u0026ldquo;Performance Expectancy\u0026rdquo; was operationalized as \u0026ldquo;How useful would [RxFill] be in your typical practice?\u0026rdquo; (Question 24, Appendix 1). As a second example, UTAUT\u0026rsquo;s \u0026ldquo;Effort Expectancy\u0026rdquo; was phrased as, \u0026ldquo;How easy do you think it would be to find and use [RxFill]in your typical practice?\u0026rdquo; (Question 28, Appendix 1).\u003c/p\u003e\u003cp\u003eParticipants also completed a brief demographic survey and System Usability Scale (SUS) survey. The entire data collection session took approximately 45 minutes and participants received \u003cspan\u003e$\u003c/span\u003e100 remuneration for their time. We observed the participant completing the steps and audio recorded the encounter. The think-aloud and interview audio recording was transcribed verbatim by a professional service.\u003c/p\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eData Analysis\u003c/h2\u003e\u003cp\u003eThe qualitative thematic analysis was conducted using NVivo 20 (NVivo, Lumivero) following the methodology of Braun and Clarke, using a codebook approach (2019).(\u003cspan additionalcitationids=\"CR28\" citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e) Two study team members (TW and AG) read the interview think-aloud commentary and interview transcripts. We developed a codebook (Appendix 2) guided generally by the constructs from the UTAUT conceptual framework (e.g., RxFill Performance Expectancy, Effort Expectancy, and Facilitating Conditions). After the first interview, we expanded the codebook to further explore UTAUT\u0026rsquo;s \u0026ldquo;Facilitating Conditions\u0026rdquo; and \u0026ldquo;Other Factors\u0026rdquo; and situate RxFill within the larger context of primary care practice\u0026mdash;1) RxFill use was likely dependent on prescribers\u0026rsquo; general clinical practice, how they anticipated patient issues, and their comfort with the EHR; as well as 2) how prescribers perceived medication adherence and how they normally engaged patients in conversations surrounding medication adherence. Together, we (TW and AG) used the codebook to synchronously code one transcript and achieve complete agreement. We then independently coded the remaining seven transcripts using deductive analysis.(\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e) Transcripts were compared to identify inter-coder reliability (average 98.94% agreement); any discordance was discussed until 100% concordance was reached between the two coders. Coded transcripts were then aggregated into themes according to the UTAUT dimensions of usability. Exemplar quotes were chosen to represent themes in the final manuscript.\u003c/p\u003e\u003cp\u003eThe SUS survey was scored and analyzed via descriptive statistics in R Statistical Software (v4.3.1; R Core Team 2021).(\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e) Generally, scores ranging between 0 to 50 are considered \u0026ldquo;Not Acceptable,\u0026rdquo; scores between 51 and 70 are considered \u0026ldquo;Marginal,\u0026rdquo; and scores greater than 70 are considered \u0026ldquo;Acceptable.\u0026rdquo;(\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e)\u003c/p\u003e\u003c/div\u003e"},{"header":"RESULTS","content":"\u003cp\u003eRecruitment emails were sent to approximately 115 individuals in [Affiliation] department of General International Medicine, with additional convenience sampling during [Affiliation] meetings and seminars. Eight primary care prescribers participated in the study (6% response rate, 1 short of a priori recruitment goal), a majority being white (87.5%) and female (87.5%). On average, participants reported 19.5 years of healthcare experience (ranging from 12 to 29 years) and were 43.5 years old (ranging from 37 to 52 years old).\u003c/p\u003e\u003cp\u003eThe SUS survey yielded an average score of 81.25, ranging from 65 to 97.5, indicating it was considered \u0026ldquo;Acceptable.\u0026rdquo;\u003c/p\u003e\u003cp\u003e\u003cb\u003eQualitative Assessment of Themes\u003c/b\u003e\u003c/p\u003e\u003cp\u003eThe think-aloud commentary and interviews shed light on several themes regarding RxFill acceptance and usability that aligned with UTAUT\u0026mdash;Usefulness of RxFill, Ease of Use of RxFill, Facilitating Conditions to Use RxFill, and Intention to Use RxFill. Exemplary quotes are provided for each theme discussed.\u003c/p\u003e\u003cp\u003eTheme: Usefulness of RxFill\u003c/p\u003e\u003cp\u003eOf note, none of the participants knew about RxFill or the new functionality prior to participating in the study. However, most of the participants stated that the dispense dates provided by RxFill were useful.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI think it\u0026rsquo;s useful for the conversation with the patient, but then also I\u0026rsquo;m sure it would be helpful for the staff to have as well, so if there\u0026rsquo;s an issue with how the prescription is written. But, it would guide how I talk to [patients] because it tells me how they\u0026rsquo;re taking the medications indirectly. \u0026ndash; Participant 1\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eMost participants stated that RxFill and the dispense dates were particularly useful for complex patients, or specific disease states such as diabetes or high cholesterol.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI think this would be a point of care if something doesn\u0026rsquo;t make sense in terms of [the patient\u0026rsquo;s] A1C or their blood pressure or their lipids, this is where I\u0026rsquo;d go. But I think albuterol would be a really good [medication] to look at for control data, although I\u0026rsquo;m very aware that people use each other\u0026rsquo;s albuterol inhalers in households. And then I think any other PRN for a condition we\u0026rsquo;re following like chronic pain, it might be useful to check this before I go in the room. \u0026ndash; Participant 8\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eParticipants also stated that the dispense dates were useful to compare against patient vitals, lab results, therapy outcomes, or other patient reports.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI think where I might use it more is not necessarily preparing [for the visit] but after I\u0026rsquo;ve talked to [the patient], or if we\u0026rsquo;re seeing [their] cholesterol is not controlled, then I can go back and ask, \u0026ldquo;are they actually, getting their medication?\u0026rdquo; Or if as I am preparing, I see that their cholesterol isn\u0026rsquo;t at goal, then this might be something would pull up and dig into. \u0026ndash; Participant 4\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eTheme: Ease of Use of RxFill\u003c/p\u003e\u003cp\u003eWhile the participants agreed that RxFill data and the dispense information was useful and easy to comprehend to approximate adherence, their responses varied on how easy RxFill was to locate and access, citing difficulties navigating the EHR to find the data.\u003c/p\u003e\u003cp\u003eFor some participants who previously had their medical assistants (MAs) and clinic staff call patients\u0026rsquo; pharmacies to obtain adherence information, RxFill was an easy, time-saving functionality.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI think it would be really helpful. I didn\u0026rsquo;t realize I could see when these pharmacies are filling things. And that used to always take me asking my MA to call the pharmacy, and they\u0026rsquo;re always busy, so I don\u0026rsquo;t do that very often unless it\u0026rsquo;s more critical like a new patient, we don\u0026rsquo;t know exactly what dose they\u0026rsquo;re on and things. So I think it would be incredibly helpful. \u0026ndash; Participant 2\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eConversely, other participants reported that RxFill, as implemented, was not easy to use and that it was cumbersome to run reports for each medication of interest.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003e[RxFill\u0026rsquo;s] useful, but it\u0026rsquo;s fairly onerous because, you would have to click on the report button and look through [each medication] every time. And to look through every medication would take you several minutes at least, and so I would probably only use it in select cases. \u0026ndash; Participant 3\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eThe way [RxFill] sits, it\u0026rsquo;s not practical at all because it takes several clicks, it takes a couple clicks to get to. And I have really small report link, and when I see a report link, it usually just means I get some sort of report that has a bunch of stuff in it. \u0026ndash; Participant 1\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eTheme: Facilitating Conditions/Personal and Work System Characteristics\u003c/p\u003e\u003cp\u003eSeveral work system and personal characteristics influenced RxFill usability. Most participants stated that their busy workload prohibited them from spending too much time on reviewing the medication list before the encounter, including the new RxFill functionality. One participant stated that, in general, they\u0026rsquo;re often running late and have less time to prep.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eIf I\u0026rsquo;m running late, I do less prep, which is arguably a terrible idea. [\u0026hellip;] But honestly, I\u0026rsquo;m usually not looking at meds before I go in the room. My medical assistant does a [medication] review, and then, when I\u0026rsquo;m going over the medical issues list in the room with the patient, then that\u0026rsquo;s when I\u0026rsquo;ll ask questions about medication adherence and tolerance. \u0026ndash; Participant 8\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eI don\u0026rsquo;t think I would look at every medication for every patient for sure. Would not have time to do that. So, I likely wouldn\u0026rsquo;t do it at every encounter, any, every visit, but it\u0026rsquo;s very easy to find and available. \u0026ndash; Participant 2\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eTheme: Intentions to Use RxFill\u003c/p\u003e\u003cp\u003eRxFill usefulness, ease of use, other facilitating conditions, and trust in the data, combined to indicate the participants\u0026rsquo; intentions to use RxFill in their actual practice. Most participants indicated that they could see themselves using RxFill and the filling data in \u0026ldquo;some\u0026rdquo; capacity, but only in certain scenarios.\u003c/p\u003e\u003cp\u003eFor example, participants reported that they could use RxFill data as a resource when triggered by another source, such as an out-of-range vital, lab, or patient report. Others mentioned that the usefulness of RxFill data was limited on its own, several even questioning the reliability of RxFill dispense date, but that the information may be helpful to create a holistic picture of a patients\u0026rsquo; medication experience.\u003cdiv class=\"BlockQuote\"\u003e\u003cp\u003e\u003cem\u003eI guess I would want to know how reliable [RxFill] is. Oftentimes, when you look at things like [the Prescription Drug Monitoring Program] for refills, sometimes it\u0026rsquo;s not accurate. So, I want to know how accurate is this? Does every pharmacy report it? Or maybe they got it at a pharmacy out of state, does it not report it then? So what pharmacies are on this? How reliable is it? \u0026ndash; Participant 2\u003c/em\u003e\u003c/p\u003e\u003cp\u003e\u003cem\u003eI feel like I\u0026rsquo;m not in a place where I trust that refill data is the only piece of data I need on adherence. This is more information than I knew I had here, and now I know how to access it. [\u0026hellip;] I think that now that I know it exists, I\u0026rsquo;ll actually access this. But I just don\u0026rsquo;t have time to do this for every single patient. \u0026ndash; Participant 8\u003c/em\u003e\u003c/p\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eOther participants stated they may look at RxFill when \u0026ldquo;in the room\u0026rdquo; with the patient in real-time but may not use RxFill to prepare for an encounter.\u003c/p\u003e\u003cp\u003e\u003cem\u003eIf someone was coming in for an annual review, when there\u0026rsquo;s so many things to go over, I probably wouldn\u0026rsquo;t [look at RxFill] beforehand. If someone was coming specifically for asthma, I might look at [RxFill] beforehand. Whereas if I\u0026rsquo;m planning an encounter, and I could look at it right away, it could be a little more real time and able to address it in real time. \u0026ndash; Participant 4\u003c/em\u003e\u003c/p\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eThese findings suggested overall support for RxFill, as implemented, being potentially useful and helpful to engage patients in conversations around medication adherence. Ultimately, however, when coupled with the limited inflow of data from community pharmacies (described in the Setting section), RxFill was not likely to be actually used in practice.\u003c/p\u003e\u003cp\u003e\u003cb\u003eIssue 1: RxFill\u0026rsquo;s Usefulness in Real-Time During Patient Encounters\u003c/b\u003e\u003c/p\u003e\u003cp\u003eParticipant Feedback\u003c/p\u003e\u003cp\u003eWhen originally proposed, we hypothesized RxFill would be most useful when anticipating patient concerns before an encounter, or when doing a chart review prior to a visit\u0026mdash;prescribers would be actively searching for and evaluating non-adherence before walking into the patient room.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan additionalcitationids=\"CR6\" citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e) Participant feedback, however, revealed that many prescribers do not have time for such an in-depth analysis within their already busy workload, a sentiment echoed in the literature.(\u003cspan additionalcitationids=\"CR36\" citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e) Additionally, one participant reported that they do not consider medication adherence at all prior to a patient encounter, instead focusing on other outcomes such as lab results, vitals, or symptoms. Once in the room with the patient, they relied on patient reports and were hesitant to utilize objective metrics of medication use such as dispense dates alone. Participants also questioned the reliability of RxFill data, based on their prior experiences with inadequate reporting from other HIT functionalities.(\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eRecommendations\u003c/p\u003e\u003cp\u003eAs designed, RxFill use is dependent upon the value prescribers ascribe to evaluating medication non-adherence.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) RxFill requires prescribers to actively retrieve data, as opposed to other functionalities that flag or alert non-adherent behaviors.(\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e) In considering best practices and current design, RxFill may be most useful when in the room with the patient as a way to quickly and objectively assess medication adherence in real-time. Prescribers should consider RxFill as another source of information when patient outcomes or reports do not align with medication therapy. Prescribers who leverage RxFill data should use their clinical and motivational interviewing skills, such as using patient-centered language and non-judgmental and non-biased tones, to engage patients in conversations around medication adherence.(\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e)\u003c/p\u003e\u003cp\u003e Participants also mentioned the utility of RxFill for other clinic staff members. For example, having RxFill information may be helpful for Medical Assistants (MA) or other clinic personnel while reviewing medication lists with patients at the beginning of an encounter.(\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e) Similarly, RxFill information may be helpful for nurses when answering patient or pharmacist questions regarding prescriptions.(\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e, \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e) Additionally, with pharmacists now often embedded into primary care or other ambulatory practices, having access to RxFill information may be helpful when conducting Comprehensive Medication Reviews (CMRs) or other medication management services.(\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e, \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e)\u003c/p\u003e\u003cp\u003e\u003cb\u003eIssue 2: RxFill Needs to be Easy and Intuitive to Access and Use\u003c/b\u003e\u003c/p\u003e\u003cp\u003eParticipant Feedback\u003c/p\u003e\u003cp\u003eParticipants emphasized that RxFill data needs to be easy to access and intuitive to interpret and use. However, most participants reported that they were unlikely to use RxFill in actual practice because it required numerous clicks and was an onerous process, especially for patients with many medications. When we began this study, we hypothesized that RxFill data and adherence assessments would be universally valuable and would be a standard of care for all medications. Practically however, given the high click burden participants stated they more likely to target RxFill use for patients with diabetes or hypertension, where medication adherence is crucial for patient outcomes.\u003c/p\u003e\u003cp\u003eRecommendations\u003c/p\u003e\u003cp\u003eThe EHR contains massive amounts of clinical information; prescribers must rapidly retrieve and use this information to make clinical decisions during a short patient encounter.(\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e) However, just because information exists within the EHR does not mean it will be used as intended.(\u003cspan additionalcitationids=\"CR17 CR18\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e) A study of drug safety alerts found that the alerts were overridden up to 96% of the time.(\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e) As evident by our findings, even a new IT that is deemed \u0026ldquo;simple,\u0026rdquo; such as RxFill, may not be used if providers 1) do not know that the information exists, 2) cannot find the information readily in the EHR, or 3) cannot quickly interpret and use the information during a patient visit.(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan additionalcitationids=\"CR51\" citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e)\u003c/p\u003e\u003cp\u003eWhen organizations are choosing to implement RxFill, they should consider a design that is intuitive and does not take any additional time beyond prescriber current practice. The new technology should be time \u0026ldquo;neutral\u0026rdquo; (meaning it does not add any additional time) or time \u0026ldquo;negative\u0026rdquo; (meaning it reduces the time necessary to complete a task).(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e, \u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e) Organizations should utilize end-user input to determine how prescribers currently access medication information to tailor RxFill to best suit their prescribers\u0026rsquo; needs.(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e) For example, where do prescribers access and review patient medications? This may provide insight on where to integrate RxFill data. Should medical assistants or nurses be responsible for reviewing RxFill information and triaging complex patients prior to the encounter? This may be helpful to standardize practice protocols or distribute tasks amongst team members. Additionally, organizations and prescribers should consider prioritizing RxFill use and assessing medication adherence for \u0026ldquo;high risk\u0026rdquo; medications or disease states.(\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e) While adherence is still critical to evaluate for patients, we recognize that time, workload, and other contextual restraints make this standard difficult to implement in practice. Therefore, it\u0026rsquo;s important to make RxFill use as easy as possible to promote its actual use.\u003c/p\u003e\u003cp\u003e\u003cb\u003eIssue 3: RxFill Use is Limited by Community Pharmacy Adoption\u003c/b\u003e\u003c/p\u003e\u003cp\u003eParticipant Feedback\u003c/p\u003e\u003cp\u003eUltimately prescribers saw the value and usefulness of RxFill, even when its actual use was somewhat undermined by its design. A major limitation to RxFill\u0026rsquo;s actual use was the lack of adoption by community pharmacies and flow of information between community pharmacies and clinic EHR.\u003c/p\u003e\u003cp\u003eRecommendations\u003c/p\u003e\u003cp\u003eWhile we recommend that community pharmacies support sending RxFill data and transactions, we recognize that barriers, such as Surescripts transaction costs, may limit implementation in practice. For RxFill to be adopted by community pharmacies, and therefore adopted by healthcare systems, organizations, and prescribers, future work should attempt to reduce barriers that limit its use.\u003c/p\u003e\u003cp\u003e\u003cb\u003eStrengths and Limitations\u003c/b\u003e\u003c/p\u003e\u003cp\u003eAs evident by these potential RxFill issues and recommendations, the central limitation of this study is the lack of broad generalizability. This study took place in one midwestern academic health system and researchers had organizational support and buy-in throughout RxFill implementation. Additionally, the usability study and qualitative interviews were limited to the perspectives of those who chose to participate, indicating the potential for selection bias (self-selection of prescribers who may have higher comfort using the EHR or interest in medication adherence). Amidst these limitations, however, the study leveraged the UTAUT framework and \u0026ldquo;think aloud\u0026rdquo; protocols to examine usability of RxFill. The study provided recommendations for organizations intending to implement RxFill to ensure health IT success.\u003c/p\u003e"},{"header":"CONCLUSIONS","content":"\u003cp\u003eThis project demonstrated the potential value of RxFill\u0026mdash;providing the right information, to the right person, at the right time. RxFill provided the \u0026ldquo;right information\u0026rdquo; by communicating accurate dispensing and medication fill information that can be utilized to inform patient adherence calculations, clinical decisions, and conversations with patients. RxFill provided this information to the \u0026ldquo;right person\u0026rdquo; by sharing information from the patient\u0026rsquo;s pharmacy to providers. Finally, RxFill provided this information at the \u0026ldquo;right time\u0026rdquo;\u0026mdash;within the EHR at the time of encounter before and during the appointment. Specifically, by delivering patient fill and adherence data at the right time, RxFill potentially facilitates conversations between patients and providers that can improve medication adherence and resulting clinical outcomes. However, this project also demonstrated key barriers in RxFill design and infrastructure that limited prescribers\u0026rsquo; intentions to use in actual practice. While we provide recommendations for other health systems, clinics, and prescribers planning to utilize RxFill, this study also highlights the importance of usability evaluation as new HIT functionalities are trialed and implemented into the standard of care.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eAPNP\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eAdvanced practice Nurse Practitioner\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eEHR\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eElectronic Health Record\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eDO\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eDoctor Osteopathic Medicine\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eHIT\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eHealth Information Technology\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eMD\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eDoctor of Medicine [MD]\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eNCPDP\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eNational Council for Prescription Drug Programs\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003ePA\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003ePhysician Assistant\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eSUS\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eSystem Usability Scale\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003cdiv class=\"DefinitionListEntry\"\u003e\u003cdiv class=\"Term\"\u003eUTAUT\u003c/div\u003e\u003cdiv class=\"Description\"\u003e\u003cp\u003eUnified Theory of Acceptance and Use of Technology\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate:\u0026nbsp;\u003c/strong\u003eEthics approval was obtained before data collection with a waiver of signed consent. This study was approved by the Health Sciences Institutional Review Board (HS IRB) at the University of Wisconsin-Madison(ID: 2023-0498). The HS IRB reviews and approves research in accordance with the laws of the United States of America and the State of Wisconsin. The IRB complies with the applicable requirements of the Department of Health and Human Services (DHHS) regulations, 45 CFR Part 46; the Food and Drug Administration (FDA) regulations, 21 CFR Parts 50, 56, 312, and 812; Veteran’s Administration (VA) Regulations pertaining to the protection of human subjects, 38 CFR Part 16; and the privacy requirements of the Health Insurance Portability and Accountability Act of 1996 implemented by 45 CFR Parts 160 and 164 (Privacy Rule).\u003c/p\u003e\n\u003cp\u003eClinical Trial Registration: Not applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u0026nbsp;\u003c/strong\u003eParticipant information sheet (waiver of signed consent) included permission to publish de-identified statements.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u0026nbsp;\u003c/strong\u003eThe datasets generated and/or analyzed during the current study are not publicly available to protect the identity of the participants but are available from the corresponding author on reasonable request and approval from UW Health.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting Interests:\u0026nbsp;\u003c/strong\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003eThis project was supported by the National Council for Prescription Drug Programs (NCPDP) Foundation. TW was supported by the University of Wisconsin Primary Care Research Fellowship, funded by grant T32HP10010 from the Health Resources and Services Administration.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors’ Contributions:\u0026nbsp;\u003c/strong\u003eTW was responsible for conceptualization, methodology, analysis, writing – original draft, project administration, and funding acquisition. AG was responsible for conceptualization, methodology, analysis, writing – review \u0026amp; editing. JS and PK were responsible for conceptualization, methodology, writing – review \u0026amp; editing.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements:\u003c/strong\u003e Chelsea Steitz and Emily Hoffins\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eBrown MT, Bussell JK. Medication adherence: WHO cares? Mayo Clin Proc. 2011;86(4):304\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBrown MT, Bussell J, Dutta S, Davis K, Strong S, Mathew S. Medication Adherence: Truth and Consequences. Am J Med Sci. 2016;351(4):387\u0026ndash;99.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eOsterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487\u0026ndash;97.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHeidenreich PA. Patient adherence: the next frontier in quality improvement. Am J Med Elsevier BV. 2004;117(2):130\u0026ndash;2.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKwan YH, Weng SD, Loh DHF, Phang JK, Oo LJY, Blalock DV, et al. Measurement Properties of Existing Patient-Reported Outcome Measures on Medication Adherence: Systematic Review. J Med Internet Res. 2020;22(10):e19179.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNguyen T-M-U, La Caze A, Cottrell N. What are validated self-report adherence scales really measuring? a systematic review. Br J Clin Pharmacol. 2014;77(3):427\u0026ndash;45.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSch\u0026ouml;nfeld MS, Pfisterer-Heise S, Bergelt C. Self-reported health literacy and medication adherence in older adults: a systematic review. BMJ Open. 2021;11(12):e056307.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHebel S, Kahn-Woods E, Malone-Thomas S, McNeese M, Thornton L, Sukhija-Cohen A, et al. Brief Report: Discrepancies Between Self-Reported Adherence and a Biomarker of Adherence in Real-World Settings. J Acquir Immune Defic Syndr. 2020;85(4):454\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKrueger KP, Berger BA, Felkey B. Medication adherence and persistence: a comprehensive review. Adv Ther. 2005;22(4):313\u0026ndash;56.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePrieto-Merino D, Mulick A, Armstrong C, Hoult H, Fawcett S, Eliasson L, et al. Estimating proportion of days covered (PDC) using real-world online medicine suppliers\u0026rsquo; datasets. J Pharm Policy Pract. 2021;14(1):113.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWatterson TL, Stone JA, Gilson A, Brown R, Xiong KZ, Schiefelbein A et al. Impact of CancelRx on discontinuation of controlled substance prescriptions [Internet]. bioRxiv. medRxiv; 2021. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://medrxiv.org/lookup/doi/\u003c/span\u003e\u003cspan address=\"http://medrxiv.org/lookup/doi/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1101/2021.01.12.21249700\u003c/span\u003e\u003cspan address=\"10.1101/2021.01.12.21249700\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWatterson TL, Stone JA, Brown R, Xiong KZ, Schiefelbein A, Ramly E et al. CancelRx: a health IT tool to reduce medication discrepancies in the outpatient setting. J Am Med Inform Assoc [Internet]. 2021; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://dx.doi.org/10.1093/jamia/ocab038\u003c/span\u003e\u003cspan address=\"10.1093/jamia/ocab038\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRupp MT, Warholak TL. Evaluation of e-prescribing in chain community pharmacy: best-practice recommendations. J Am Pharm Assoc. 2008;48(3):364\u0026ndash;70.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNational Council for Prescription Drug Programs. NCPDP Electronic Prescribing Standards [Internet]. 2014. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.ncpdp.org/NCPDP/media/pdf/NCPDPEprescribingBasics.pdf\u003c/span\u003e\u003cspan address=\"https://www.ncpdp.org/NCPDP/media/pdf/NCPDPEprescribingBasics.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSCRIPT Implementation Recommendations [Internet]. National Council for Prescription Drug Programs. 2023 May. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.ncpdp.org/NCPDP/media/pdf/SCRIPT-Implementation-Recommendations.pdf\u003c/span\u003e\u003cspan address=\"https://www.ncpdp.org/NCPDP/media/pdf/SCRIPT-Implementation-Recommendations.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHarrison MI, Koppel R, Bar-Lev S. Unintended Consequences of Information Technologies in Health Care\u0026ndash;An Interactive Sociotechnical Analysis [Internet]. Vol. 14, Journal of the American Medical Informatics Association. 2007. pp. 542\u0026ndash;9. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://dx.doi.org/10.1197/jamia.m2384\u003c/span\u003e\u003cspan address=\"10.1197/jamia.m2384\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarsh B-T, Weinger MB, Abbott PA, Wears RL. Health information technology: fallacies and sober realities. J Am Med Inf Assoc. 2010;17(6):617\u0026ndash;23.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarsh B-T. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care. 2004;13(5):388\u0026ndash;94.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarsh B-T, Holden RJ, Alper SJ, Or CKL. A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Qual Saf Health Care. 2006;15(Suppl 1):i59\u0026ndash;65.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLi AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, et al. Integrating usability testing and think-aloud protocol analysis with near-live clinical simulations in evaluating clinical decision support. Int J Med Inf. 2012;81(11):761\u0026ndash;72.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHealth. Information Exchange (HEI) [Internet]. [cited 2024 Feb 8]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.uwhealth.org/health-information-exchange-hie\u003c/span\u003e\u003cspan address=\"https://www.uwhealth.org/health-information-exchange-hie\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMerchant P. The epic. Verona, WI: Routledge; 2017.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eVenkatesh V, Morris MG, Davis GB, Davis FD. User Acceptance of Information Technology: Toward a Unified View. Miss Q. 2003;27(3):425\u0026ndash;78.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHolden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. J Biomed Inf. 2010;43(1):159\u0026ndash;72.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNielsen J. How Many Test Users in a Usability Study? [Internet]. Nielsen Norman Group. 2012 [cited 2025 Jul 3]. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.nngroup.com/articles/how-many-test-users/\u003c/span\u003e\u003cspan address=\"https://www.nngroup.com/articles/how-many-test-users/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBraun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual Res Sport Exerc Health. 2021;13(2):201\u0026ndash;16.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eQSR International (Lumivero). NVivo 12. 2017.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBraun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77\u0026ndash;101.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBraun V, Clarke V, Hayfield N, Terry G. Thematic Analysis. Handbook of Research Methods in Health Social Sciences. Singapore: Springer Singapore; 2019. pp. 843\u0026ndash;60.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCrotty M, Crotty MF. The Foundations of Social Research: Meaning and Perspective in the Research Process. SAGE; 1998. p. 248.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRezigalla AA. Observational Study Designs: Synopsis for Selecting an Appropriate Study Design. Cureus. 2020;12(1):e6692.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBrooke J. SUS: A quick and dirty usability scale. Usability Eval Ind. 1995;189.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eR Core Team. R: A Language and Environment for Statistical Computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing. 2021. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.R-project.org/\u003c/span\u003e\u003cspan address=\"https://www.R-project.org/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBasu S. Accurately estimating medication non-adherence through patient self-report: possibilities and limitation of a new scale. Curr Med Res Opin. 2021;37(8):1349\u0026ndash;51.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLee M, Leonard C, Greene P, Kenney R, Whittington MD, Kirsh S, et al. Perspectives of VA primary care clinicians toward electronic consultation-related workload burden: A qualitative analysis: A qualitative analysis. JAMA Netw Open. 2020;3(10):e2018104.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRittenberg E, Liebman JB, Rexrode KM. Primary care physician gender and electronic health record workload. J Gen Intern Med. 2022;37(13):3295\u0026ndash;301.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAgarwal SD, Pabo E, Rozenblum R, Sherritt KM. Professional dissonance and burnout in primary care: A qualitative study: A qualitative study. JAMA Intern Med. 2020;180(3):395\u0026ndash;401.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTung F-C, Chang S-C, Chou C-M. An extension of trust and TAM model with IDT in the adoption of the electronic logistics information system in HIS in the medical industry. Int J Med Inf. 2008;77(5):324\u0026ndash;35.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLuo J, Wong R, Mehta T, Schwartz JI, Epstein JA, Smith E, et al. Implementing real-time prescription benefit tools: Early experiences from 5 academic medical centers. Healthc (Amst). 2023;11(2):100689.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eZaugg V, Korb-Savoldelli V, Durieux P, Sabatier B. Providing physicians with feedback on medication adherence for people with chronic diseases taking long-term medication. Cochrane Database Syst Rev. 2018;1(1):CD012042.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKane-Gill SL, O\u0026rsquo;Connor MF, Rothschild JM, Selby NM, McLean B, Bonafide CP, et al. Technologic distractions (part 1): Summary of approaches to manage alert quantity with intent to reduce alert fatigue and suggestions for alert fatigue metrics. Crit Care Med. 2017;45(9):1481\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcQuaid EL, Landier W. Cultural Issues in Medication Adherence: Disparities and Directions. J Gen Intern Med. 2018;33(2):200\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eReedy AB, Yeh JY, Nowacki AS, Hickner J. Patient, physician, medical assistant, and office visit factors associated with medication list agreement. J Patient Saf. 2016;12(1):18\u0026ndash;24.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eNorful A, Martsolf G, de Jacq K, Poghosyan L. Utilization of registered nurses in primary care teams: A systematic review. Int J Nurs Stud. 2017;74:15\u0026ndash;23.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCelio J, Ninane F, Bugnon O, Schneider MP. Pharmacist-nurse collaborations in medication adherence-enhancing interventions: A review. Patient Educ Couns. 2018;101(7):1175\u0026ndash;92.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eYoo A, Fennelly JE, Renauer MM, Coe AB, Choe HM, Marshall VD et al. Comprehensive medication review service by embedded pharmacists in primary care: Innovations and impact. J Am Pharm Assoc (2003). 2022;62(2):580\u0026ndash;587.e1.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGernant SA, Bacci JL, Upton C, Ferreri SP, McGrath S, Chui MA, et al. Three opportunities for standardization: A literature review of the variation among pharmacists\u0026rsquo; patient care services terminology. Res Social Adm Pharm. 2020;16(6):766\u0026ndash;75.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGawande A. Why Doctors Hate Their Computers. The New Yorker [Internet]. 2018; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers\u003c/span\u003e\u003cspan address=\"https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarsh B-T. Clinical practice improvement and redesign: how change in workflow can be supported by clinical decision support. Rockville, MD: Agency for Healthcare Research and Quality [Internet]. 2009;200943. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://healthit.ahrq.gov/sites/default/files/docs/biblio/09-0054-EF-Updated_0.pdf\u003c/span\u003e\u003cspan address=\"https://healthit.ahrq.gov/sites/default/files/docs/biblio/09-0054-EF-Updated_0.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eJaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inf. 2009;78(5):340\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRatwani RM, Savage E, Will A, Arnold R, Khairat S, Miller K, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inf Assoc. 2018;25(9):1197\u0026ndash;201.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBrunner J, Chuang E, Goldzweig C, Cain CL, Sugar C, Yano EM. User-centered design to improve clinical decision support in primary care. Int J Med Inf. 2017;104:56\u0026ndash;64.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKarsh B-T, Escoto KH, Beasley JW, Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon. 2006;37(3):283\u0026ndash;95.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eGardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inf Assoc. 2019;26(2):106\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eWoodward M, Dixon-Woods M, Randall W, Walker C, Hughes C, Blackwell S et al. How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study. BMJ Qual Saf [Internet]. 2023 Dec 12 [cited 2023 Dec 18]; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://qualitysafety.bmj.com/content/early/2023/12/12/bmjqs-2023-016196?utm_campaign=Research%20papers\u0026amp;utm_medium=email\u0026amp;_hsmi=286361573\u0026amp;_hsenc=p2ANqtz-9JhxxIS9NwPR7FWomvykN6DPuDul4XbMyl_PVoj61UI_krOXq7EiYzbFL0lSn_DfrXv8FAdNXYKLA53_A8xIf3-aMIhQ\u0026amp;utm_content=286361573\u0026amp;utm_source=hs_email\u003c/span\u003e\u003cspan address=\"https://qualitysafety.bmj.com/content/early/2023/12/12/bmjqs-2023-016196?utm_campaign=Research%20papers\u0026amp;utm_medium=email\u0026amp;_hsmi=286361573\u0026amp;_hsenc=p2ANqtz-9JhxxIS9NwPR7FWomvykN6DPuDul4XbMyl_PVoj61UI_krOXq7EiYzbFL0lSn_DfrXv8FAdNXYKLA53_A8xIf3-aMIhQ\u0026amp;utm_content=286361573\u0026amp;utm_source=hs_email\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inf. 2020;11(1):46\u0026ndash;58.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Medication adherence, Pharmacy, Usability","lastPublishedDoi":"10.21203/rs.3.rs-5485583/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5485583/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e\u003cp\u003eOver 50% of patients do not take their medications as prescribed or are non-adherent. Primary care providers are well positioned to address non-adherence. However, current methods for obtaining adherence are unreliable. RxFill integrates prescription medication fill status and dispense dates from community pharmacies into clinic electronic health records (EHRs). The goal of this study was to examine RxFill usability during a simulated case with primary care providers.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e\u003cp\u003eThe study took place at an academic health system that implemented RxFill in June 2022. Participants were asked to review a simulated patient chart and \u0026ldquo;think aloud.\u0026rdquo; After, a semi-structured interview elicited attitudes towards RxFill, including usability. Participants also completed the System Usability Scale (SUS), which scores the functionality on a scale of 0-100. Think-aloud commentary and interview transcripts were analyzed via qualitative content analysis.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e\u003cp\u003eEight providers participated in the study. The average SUS score was 81.25, generally considered \u0026ldquo;Acceptable.\u0026rdquo; Qualitative themes included RxFill usefulness, ease of use, and fit into current workload. Participants reported RxFill data was useful to estimate patient adherence, however, running reports for each medication was time intensive. Participants had minimal time to review patient charts prior to appointments, which would limit their ability to use RxFill in practice.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e\u003cp\u003eRxFill data, although useful, was not entirely easy to use and did not fit into participants\u0026rsquo; current workload. Institutions implementing RxFill should make dispense dates and fill indicators easy to access and consider workload and integration into other health IT systems. Additionally, RxFill use is limited by community pharmacy adoption and the ability to send fill data.\u003c/p\u003e","manuscriptTitle":"Examining Usability of RxFill: Integrating Health IT to Support Medication Adherence","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-18 09:01:45","doi":"10.21203/rs.3.rs-5485583/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-24T14:01:05+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-25T17:40:28+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"118923828942134915373434540430592036373","date":"2025-08-19T14:00:53+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-08-16T00:02:26+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"183238496080091333533035926717322542766","date":"2025-08-15T14:59:50+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-08-15T14:51:39+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-07-10T16:58:46+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2025-07-07T19:45:38+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"445f6711-08c0-4a8c-a788-cc7f63bd2529","owner":[],"postedDate":"August 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-04-20T16:12:22+00:00","versionOfRecord":{"articleIdentity":"rs-5485583","link":"https://doi.org/10.1186/s12911-026-03496-z","journal":{"identity":"bmc-medical-informatics-and-decision-making","isVorOnly":false,"title":"BMC Medical Informatics and Decision Making"},"publishedOn":"2026-04-16 15:58:56","publishedOnDateReadable":"April 16th, 2026"},"versionCreatedAt":"2025-08-18 09:01:45","video":"","vorDoi":"10.1186/s12911-026-03496-z","vorDoiUrl":"https://doi.org/10.1186/s12911-026-03496-z","workflowStages":[]},"version":"v1","identity":"rs-5485583","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5485583","identity":"rs-5485583","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}