Perspectives on Performance Measurement Pressure and Telehealth Utilization in Free and Charitable Clinics: A Qualitative Interview Study

Perspectives on Performance Measurement Pressure and Telehealth Utilization in Free and Charitable Clinics: A Qualitative Interview Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Perspectives on Performance Measurement Pressure and Telehealth Utilization in Free and Charitable Clinics: A Qualitative Interview Study Ashley Parks, Andrew Wear, Julie Sakowski, Ian Russell, Danielle Nunnery This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4714037/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background Free and charitable clinics, essential for providing healthcare to uninsured and underinsured populations, face significant performance measurement pressures. These pressures, derived from the need to demonstrate quality and efficiency, may hinder the adoption of telehealth services due to a dearth of resources to meet the demands met by in-person data collection. This study explores manager perspectives on how performance measurement pressure impacts the implementation and sustainability of telehealth. Methods An interdisciplinary research team conducted 13 key informant interviews with leaders from free and charitable clinics. The interviews, part of a larger research effort utilizing the Consolidated Framework for Implementation Research, aimed to understand free and charitable clinic implementation of telehealth during the COVID-19 pandemic. Interviews were recorded, transcribed, and analyzed using MaxQDA 22.0. Fundamental attitudes and themes regarding telehealth implementation and utilization were elicited using a qualitative descriptive technique with all constructs, including performance measurement pressure, being coded according to model definitions. The research team utilized the Guidance for publishing qualitative research in informatics checklist for result reporting. Results An unexpected but prominent theme emerged: performance measurement pressure may be a limitation to wide scale telehealth implementation. All 13 interviewees highlighted this pressure as a notable concern. Performance metrics, often requiring in-person visits, were seen as barriers to the widespread adoption of telehealth. Some managers viewed these metrics as necessary for securing funding and ensuring quality but also reported that the current metrics were incompatible with telehealth, which posed challenges for sustainable telehealth integration. Conclusions The study reveals that while telehealth could enhance care access for underserved populations, existing performance measurement frameworks hinder its adoption in free and charitable clinics. To support sustainable telehealth integration, there is a need for flexible performance metrics that accommodate telehealth’s unique capabilities. Findings from this study suggest that policymakers and accrediting bodies should consider revising quality metrics to better accommodate telehealth modalities. Future research should focus on developing specific telehealth performance indicators and data collection methods that do not rely solely on in-person visits. Addressing these issues is crucial for improving healthcare access and quality in resource-constrained settings. Trial registration Not applicable Figures Figure 1 Contributions to the Literature This study fills a gap in the literature by exploring the impact of performance measurement pressure on telehealth utilization in free and charitable clinics (FCCs). Using the Consolidated Framework for Implementation Research (CFIR), it provides novel insights into unique barriers faced by FCCs. The qualitative interviews offer empirical evidence from free and charitable clinic leaders, enhancing the depth of understanding regarding challenges in this setting. The findings highlight the need for policymakers to revise performance metrics to support telehealth integration in FCCs, addressing the dual pressures of administrative burden and resource allocation in these essential healthcare providers. Introduction Although free and charitable clinics (FCCs) generally operate outside of traditional fee for service and capitated billing structures, these organizations typically coordinate, code, and document health services within or above industry standards ( 1 ). Specifically, free and charitable clinics participate in a national quality measures reporting program, similar to those utilized across other ambulatory settings. This study explores perspectives regarding performance measurement pressure as well as data collection and reporting requirements in FCCs as a barrier to the utilization of telehealth services due to the inherent resource needs associated with capturing data in person for many performance metrics. This pressure often stems from the need to demonstrate efficiency, effectiveness to stakeholders and comply with regulations. However, the current health system’s reliance on in-person data collection may discourage clinics from fully embracing telehealth solutions, as remote consultations may not easily align with or support meeting traditional quality metrics ( 2 ). Consequently, this limitation in addition to other factors such as federal and state policies and patient preferences ( 3 ) could impede the integration of telehealth into clinic operations, potentially hindering access to care for underserved populations who could benefit from its convenience and accessibility. Performance Measurement Pressure Defined The Consolidated Framework for Implementation Research (CFIR) contains several evidence-based constructs designed to assist implementation science researchers with understanding, characterizing, and contextualizing factors impacting the implementation of new processes and technologies. One of those constructs, performance measurement pressure is defined as “Quality or benchmarking metrics or established service goals that drive implementation and/or delivery of the innovation” ( 4 ). This construct refers specifically to the impact of outside requirements including data reporting and evaluation on the feasibility, choice to implement, and success of an implementation ( 5 ). According to the CFIR, performance measurement pressure can impact an implementation and an organization’s operations if the “the threat or reality of public reporting may motivate Inner Settings, especially late-adopters, to implement an innovation in an effort not to look bad compared to their competitors''. However, in some cases “public reporting can also have a negative influence if there is an adversarial relationship between the reporting entity and the Inner Setting” ( 4 ). In the case of this study, the inner setting studied includes a network of independently operated free and charitable clinics working to implement and utilize telehealth services. Free and Charitable Clinics (FCCs) in the United States Free and charitable clinics are essential healthcare providers offering medical services to uninsured and underinsured populations in the United States. Unlike traditional healthcare facilities, these clinics do not operate under fee-for-service or capitated billing structures. Instead, they rely on donations, grants, and volunteer support to function. These clinics provide a wide range of services, including primary care, dental care, mental health services, and sometimes specialty care, often at no or minimal cost to patients ( 1 ). Many clinics benefit from in-kind donations such as medical supplies, equipment, and professional volunteer services, which significantly reduce operational costs (6). Despite limited financial resources, FCCs strive to maintain high standards of care and often participate in national quality measures reporting programs. This commitment to quality is crucial for demonstrating their value to funders and ensuring continued support. The operational model of these clinics emphasizes community-based care, targeting underserved populations who might otherwise lack access to essential health services (7). Exploring Performance Measurement in the Free and Charitable Clinic Environment Performance measurement in healthcare is a critical component for ensuring the delivery of high-quality care and improving patient outcomes. In recent years, the emphasis on performance metrics has extended beyond traditional healthcare settings to include FCCs, which serve as vital safety nets for underserved populations. According to the National Association of Free and Charitable Clinics (NAFC), these clinics provided nearly 6 million patient visits in 2022 ( 1 ). FCCs often operate with constrained budgets, relying heavily on volunteers and donations, which can complicate efforts to implement and maintain rigorous performance measurement systems. The pressure to measure and report performance can stem from multiple sources, including funding agencies, accreditation bodies, and internal organizational goals. Quality metrics, such as patient satisfaction, clinical outcomes, and operational efficiency, are crucial for demonstrating value and securing ongoing support (8). However, the resource-intensive nature of performance measurement can place significant strain on clinics that are already operating at capacity. This study explores perspectives regarding performance measurement pressure in FCCs as a barrier to the utilization of telehealth services and summarizes the challenges unique to this particular setting. The FCC environment was originally selected for this research study due to these unique constraints and the potential opportunity to explore key factors influencing the implementation and utilization of telehealth in a setting not dependent on traditional funding sources. Recently, not-for-profit organizations relying on funding from grants and donations have undergone significant scrutiny regarding the amount of their funds used for operations compared to the amount used for overhead costs (9). Performance measurement as a way to hold organizations accountable for the quality of services provided has been a concept throughout healthcare for quite some time (10), but this concept has recently gained focus with the sudden shift to telehealth placing emphasis on patient reported outcomes and introducing limiting factors such as time (11). FCCs not only contend with these complex needs to demonstrate the need for continued funding but also embrace the complex needs of the communities they serve. Clinics monitor outcomes for chronic conditions such as diabetes, hypertension, asthma, etc. and track indicators such as HbA1c levels in diabetic patients, blood pressure readings, and asthma control, the rate of vaccinations, cancer screenings (such as mammograms, colonoscopies, etc.), and other preventive services provided to patients. Methods In the spring of 2023, an interdisciplinary research team completed 13 key informant interviews with FCC leaders as part of a broader effort to utilize the Consolidated Framework on Implementation Research (CFIR) understand and contextualize the experience of implementing telehealth services during the height of the COVID-19 pandemic (12). The qualitative study exploring perspectives of clinic managers on telehealth utilization was conducted to supplement and provide context to quantitative data collected during a broader research effort to characterize telehealth implementation and utilization in FCCs (13). Qualitative research methods, particularly in-depth interviews, are well-suited to explore the nuanced experiences of clinic staff and administrators regarding performance measurement. Previous studies have highlighted the importance of contextual factors in performance measurement, suggesting that a one-size-fits-all approach may not be feasible (14). Recruitment and Sample FCC clinic directors in one state were invited to complete the initial survey. Upon completing the survey, participants were invited to participate in key informant interviews. 20 survey respondents indicated initial interest to participate. Ultimately, 12 clinic leaders and 1 administrator from a state professional association completed interviews. All participants provided informed consent and were compensated $ 99 for their time. The sample size of 13 interviews was considered sufficient given the minimum sample threshold of 12 interviews frequently discussed in the literature (15–17). Theoretical saturation was achieved from both a “coded saturation” and “meaning saturation”(18) perspective over the course of the interviews. Data Collection Interviews were conducted via Zoom in March and April 2023 and included 10 clinic managers from sites that implemented telehealth during the pandemic, 2 clinic managers from clinics that did not implement telehealth services due to various barriers discussed, and 1 representative from a state-level professional association. The interviews were recorded and the audio transcriptions, in the form of Web Video Text Tracks (WebVTT) files, were captured and uploaded into MaxQDA 22.0 qualitative analysis software. Data Analysis MaxQDA was used to clean and edit the transcripts, which were then coded and analyzed to identify common themes, elements, and attributions. The study employed thematic analysis and grounded theory to explore key themes based on the Consolidated Framework for Implementation Research (CFIR), aiming to understand telehealth implementation and use in free and charitable clinics during and after the COVID-19 pandemic. Fundamental attitudes and themes regarding telehealth implementation and utilization were elicited using a qualitative descriptive technique with all constructs, including performance measurement pressure, being coded according to the model definitions. The research team used the key tenets of the CFIR as the initial framework for coding and interpreting the interview transcripts within MaxQDA. A team of four researchers coded the interviews, reviewing for CFIR constructs, identifying other relevant key themes through open coding, and ensuring agreement on the coded segments. The coded data documented key themes related to the CFIR and created open categories, axial codes, and key themes regarding telehealth implementation and utilization explored during the key informant interviews. The coded segments analyzed in this study were part of a comprehensive effort to code all interviews using the CFIR framework. After the initial coding, these segments were reviewed to assess intercoder consistency (ICC) (19) on the main themes and to ensure a shared understanding and application of CFIR constructs for categorizing and interpreting the data. Utilizing a real-time polling system, three reviewers evaluated a sample of 20 quotes from the clinic leadership interviews, aligning each quote with the appropriate CFIR construct or theme as verified by the team. The research team utilized the Guidance for publishing qualitative research in informatics checklist for result reporting (20) (Additional file 2). Results An unexpected theme emerged specific to the CFIR construct of performance measurement pressure from the key informant interviews. While the team anticipated a variety of barriers, facilitators, challenges, and other factors to be discussed in addition to the key contracts of the CFIR model, the high intensity of comments and concern regarding performance measurement obligations as a defining limitation for telehealth as a care modality was not anticipated. This theme of performance measurement as a primary concern was present across all (n = 13, 100%) key informant interviews as a concern and key consideration impacting perspectives regarding implementation and long-term sustainable use of telehealth services. While some clinic managers reported performance measurement pressure as a primary barrier to the widespread implementation of telehealth services at the clinic site, the discussion primarily focused on how current performance measurement elements, such as quality measures, could only be achieved through the delivery of in-person clinical care visits. Table 1 featured in Additional File 1 provides a selection of coded extracts from the key informant interviews that illustrate the prominent theme of performance measurement pressure as a primary concern for FCC leaders implementing telehealth services. These extracts are broken down into the following sub-themes or axial codes as shown below in Fig. 1 . Axial Codes and Key Themes Identified Within the CFIR Construct of Performance Measurement Pressure: Performance measurement pressure and barriers meeting individual quality measure s due to the virtual nature of telehealth: These extracts specifically highlight performance measurement pressure as it relates to difficulties capturing, measuring, or reporting necessary data required to meet specific performance metrics such as management of hypertension or conducting a diabetic foot exam. Performance measurement pressure and barriers capturing data and documenting general quality outcomes across patient populations via telehealth: These coded extracts specifically highlight clinic managers’ perspectives regarding general concerns regarding measuring clinic performance that are not specific to challenges surrounding a specific quality measure. High quality healthcare services & rigorous standards for performance measurement as a priority in FCCs: These extracts showcase the perspectives of clinic managers regarding the culture of performance measurement and quality focus present in FCCs during the COVID-19 pandemic. Discussion While free and charitable clinics operate under different constraints compared to other not for profit and for profit healthcare providers (21), these organizations face significant financial and performance measurement pressure. The insights gained from this study have the potential to inform policy and practice, helping to develop tailored and supportive approaches to performance measurement in FCCs. By understanding the perspectives of those on the front lines, stakeholders can better design systems that enhance quality without overwhelming the limited resources of these essential healthcare providers. The study illustrates how the emphasis on quality metrics, which often necessitate in-person clinic visits for data collection, can inadvertently constrain the adoption and effective use of telehealth services. The findings reveal that while telehealth offers significant potential to enhance access to care, especially for underserved populations, the rigid requirements of certain performance metrics pose substantial challenges. Clinic managers reported that metrics focusing on patient satisfaction, clinical outcomes, and process adherence frequently demand physical presence, making it difficult to fully leverage the flexibility and reach of telehealth. This tension underscores a critical gap in current performance measurement frameworks, which are not always adaptable to the nuances of telehealth modalities. Moreover, the study highlights that the resource limitations endemic to FCCs exacerbate these challenges. Managers expressed concerns about the additional administrative burden and potential resource reallocation required to meet performance measurement standards while trying to expand telehealth services. This dual pressure can lead to a paradox where efforts to enhance care accessibility through telehealth are hindered by the very mechanisms intended to ensure care quality. To support the sustainable integration of telehealth in FCCs, there is a need for more flexible and inclusive performance measurement approaches. Policymakers and accrediting bodies should consider revising quality metrics to accommodate telehealth's distinct capabilities and limitations. This may involve developing new metrics that capture the quality of telehealth interactions and outcomes or modifying existing metrics to better reflect the hybrid nature of modern healthcare delivery. Future research should focus on identifying specific performance indicators that align with telehealth services and exploring innovative data collection methods that do not rely solely on in-person visits. Longitudinal studies could provide deeper insights into how performance measurement pressures evolve as telehealth becomes more embedded in healthcare practice. Limitations This study faced several limitations typical of qualitative research involving key informant interviews. As is common in qualitative research, this study employed a small sample size, which may limit the generalizability of the results to all free and charitable clinics (17). The potential for self-report and social desirability bias is high, given the lack of anonymity during Zoom interviews. Additionally, the unique circumstances of the COVID-19 pandemic created an atypical implementation environment for FCCs, which may not be generalizable to other time periods or settings. Another significant limitation is response bias. With only 12 out of 66 eligible clinic sites participating, the study might not capture the perspectives of those who did not prioritize telehealth or lacked the time to engage in the research (22). Conclusion This study underscores the importance of developing and revising performance measurement frameworks to better support the integration of telehealth in FCCs. By aligning quality metrics with the realities of telehealth, stakeholders can ensure that these clinics can fully exploit the potential of telehealth to improve care access and quality for underserved populations. Addressing these challenges is crucial for the continued evolution and effectiveness of healthcare delivery in resource-constrained settings. Abbreviations FCC(s) - Free and Charitable Clinic(s) CFIR - Consolidated Framework for Implementation Research Declarations Acknowledgements The study was approved by Appalachian State University’s Institutional Review Board (Study # HS-22-39). Informed consent was obtained from all interviewed in this project. All participants provided consent for publication. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The research team would like to acknowledge the participation and support of the North Carolina Association of Free and Charitable Clinics (NCAFCC) and the NCAFCC member clinics and leadership who participated in the key informant interviews. Their candor, expertise, and openness made this manuscript possible. Funding The author(s) received support for the research and publication of this article through a generous grant from the Appalachian Institute for Health and Wellness, formerly known as the Blue Cross North Carolina Institute for Health and Human Services (IHHS). The Implementation and Use of Telehealth by North Carolina Free and Charitable Clinics in the COVID 19 Era project was an interdisciplinary research project funded through a 2022-2023 Health Interdisciplinary Research Support Allocations (HISA) grant. Author Information Authors and Affiliations Appalachian State University, Beaver College of Health Science, Department of Nutrition and Healthcare Management, 1179 State Farm Rd, Boone, NC 28607 Ashley Parks, Andrew Wear, Julie Sakowski, Ian Russell, Danielle Nunnery Authors' contributions Authors represent an interdisciplinary collaboration within the Appalachian State University Beaver College of Health Sciences. The lead author, Ashley Parks, directed the project, led the key informant interviews of clinic managers with the active participation of all members of the interdisciplinary research team and leading the coding and analysis. Ashley Parks, Julie Sakowski, and Danielle Nunnery obtained funding for this effort through an internal interdisciplinary grant and created the initial resource proposal, Institutional Review Board documentation, and recruitment documentation. Andrew Wear, Julie Sakowski, and Danielle Nunnery worked with Dr. Parks to create an interview guide de novo and pilot test the interview guide prior to the interviews. Ashley Parks, Andrew Wear, Danielle Nunnery, and Ian Russell participated in the coding and cleaning of interview transcripts with Ashley Parks, Danielle Nunnery, Julie Sakowski, and Andrew Wear participating in coding validation through iterative comparison and polling exercises. Ashley Parks created the figure and table shared based on the team’s collective analysis. Each team member brought unique perspectives and expertise to the study, enriching the data collection process. All authors were involved in conducting interviews, ensuring a comprehensive and nuanced understanding of the participants' perspectives. All authors have participating in the writing and review of this manuscript and approved the publication. Corresponding Author Correspondence to Ashley Parks at [email protected] Ethics approval and consent to participate The study was approved by Appalachian State University’s Institutional Review Board (Study # HS-22-39). Informed consent was obtained from all interviewed in this project. Consent for publication All participants gave informed consent for their de-identified data to be published. Availability of data and materials The coded extracts featured in the article are included in Table 1. Additional data supporting the findings of this study are available from the corresponding author upon reasonable request. Competing interests The authors have no competing interests to disclose. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. References Our Impact | National Association of Free & Charitable Clinics [Internet]. [cited 2023 Sep 26]. Available from: https://nafcclinics.org/our-impact/ Tabaeeian RA, Hajrahimi B, Khoshfetrat A. A systematic review of telemedicine systems use barriers: primary health care providers’ perspective. J Sci Technol Policy Manag. 2022 Jan 1;15(3):610–35. Lin CCC, Dievler A, Robbins C, Sripipatana A, Quinn M, Nair S. Telehealth In Health Centers: Key Adoption Factors, Barriers, And Opportunities. Health Aff (Millwood). 2018 Dec;37(12):1967–74. Updated CFIR Constructs – The Consolidated Framework for Implementation Research [Internet]. [cited 2024 Jun 3]. Available from: https://cfirguide.org/constructs/ Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022 Oct 29;17(1):75. Funding | Bureau of Primary Health Care [Internet]. [cited 2024 Jun 3]. Available from: https://bphc.hrsa.gov/funding Darnell JS. Free clinics in the United States: a nationwide survey. Arch Intern Med. 2010 Jun 14;170(11):946–53. Berenson RA, Kaye DR. Grading a physician’s value--the misapplication of performance measurement. N Engl J Med. 2013 Nov 28;369(22):2079–81. Harris EE, Neely DG, Parsons LM. Nonprofit Performance Measurement and Reporting: Looking Forward. J Gov Nonprofit Account. 2022 Dec 1;11(1):51–8. Grants NRC (US) P on PM and D for PHPP. HEALTH PERFORMANCE MEASUREMENT IN THE PUBLIC SECTOR. In: Measuring Health Performance in the Public Sector: A Summary of Two Reports [Internet]. National Academies Press (US); 1999 [cited 2024 Jun 3]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK224448/ Tiem JV, Wirtz E, Suiter N, Heeren A, Fuhrmeister L, Fortney J, et al. The Implementation of Measurement-Based Care in the Context of Telemedicine: Qualitative Study. JMIR Ment Health. 2022 Nov 24;9(11):e41601. Parks A. Free and Charitable Clinic Perspectives on the Implementation and Utilization of Telehealth Services During the COVID-19 Pandemic - Ashley V. Parks, Julie A. Sakowski, Andrew G. Wear, Ian Russell, Danielle Nunnery, 2023 [Internet]. [cited 2023 Dec 19]. Available from: https://journals-sagepub-com.proxy006.nclive.org/doi/10.1177/21501319231213783 Sakowski JA, Parks A, Nunnery D, Wear A. Free and Charitable Clinic Telehealth Adoption and Utilization During the COVID-19 Era: The North Carolina Experience. Telemed Rep. 2023;4(1):215–26. Smith PC, Mossialos E, Papanicolas I. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Health Syst Health Wealth Soc Well- Assess Case Invest Health Syst. 2010 Jul 6;247–80. Braun V, Clarke V. (Mis)conceptualising themes, thematic analysis, and other problems with Fugard and Potts’ (2015) sample-size tool for thematic analysis. Int J Soc Res Methodol. 2016 Nov 1;19(6):739–43. Fugard AJB, Potts HWW. Supporting thinking on sample sizes for thematic analyses: a quantitative tool. Int J Soc Res Methodol. 2015 Nov 2;18(6):669–84. Vasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. 2018 Nov 21;18(1):148. Aldiabat K, Navenec CLL. Data Saturation: The Mysterious Step In Grounded Theory Method. Qual Rep. 2018 Jan 26;23(1):245–61. O’Connor C, Joffe H. Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines. Int J Qual Methods. 2020 Jan 1;19:1609406919899220. Ancker JS, Benda NC, Reddy M, Unertl KM, Veinot T. Guidance for publishing qualitative research in informatics. J Am Med Inform Assoc JAMIA. 2021 Sep 19;28(12):2743–8. WMJ [Internet]. [cited 2024 Jun 3]. The Safety Net’s Safety Net: Understanding the Crucial Role of Free Clinics in Cardiovascular Care. Available from: https://wmjonline.org/123no1/zellmer/ Leung L. Validity, reliability, and generalizability in qualitative research. J Fam Med Prim Care. 2015 Sep;4(3):324. Additional Declarations No competing interests reported. Supplementary Files AdditionalFile1Table1SelectedCodedExtracts.docx AdditionalFile2ChecklistforQualitativeResearch.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4714037","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":338588490,"identity":"acfa937b-703f-4d3f-81ba-cd40f629084d","order_by":0,"name":"Ashley Parks","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYHACZhAhhyxiQJQWYzDzAEQ1cVoSG4jWYs5+9rHhj5rD6f2zew9//lDzR46BvXmbBD4tlj3pxsk8xw7nzrhzLk3iwDEDYwaeY2V4tRgcSGM+zMCWlttwI8eM4QCbQWKDRI4Zfi3nnzEf/PEvLV3+Ro7xhwP/DOob5N8Q0HIjjTmBt80mweBGjoHEwTaDBAYJHvxaLGc8Yzbm7bMx3Ajyy9k+Y8M2nrRiC3xazPnTmCV/fJOQl7vde/hDxTc5eX72wxtv4HUYnCXBA6HZ8CnHrmUUjIJRMApGAToAANbUSGer7l3zAAAAAElFTkSuQmCC","orcid":"","institution":"Appalachian State University","correspondingAuthor":true,"prefix":"","firstName":"Ashley","middleName":"","lastName":"Parks","suffix":""},{"id":338588492,"identity":"ff98abe4-ae1a-481d-b9aa-aea4d73fe6fb","order_by":1,"name":"Andrew Wear","email":"","orcid":"","institution":"Appalachian State University","correspondingAuthor":false,"prefix":"","firstName":"Andrew","middleName":"","lastName":"Wear","suffix":""},{"id":338588493,"identity":"5d1c7fe0-bd84-47f2-a725-b154c7dfb45a","order_by":2,"name":"Julie Sakowski","email":"","orcid":"","institution":"Appalachian State University","correspondingAuthor":false,"prefix":"","firstName":"Julie","middleName":"","lastName":"Sakowski","suffix":""},{"id":338588496,"identity":"c4f3a0c1-b3a5-4bbb-b9f6-64278cc28c35","order_by":3,"name":"Ian Russell","email":"","orcid":"","institution":"Appalachian State University","correspondingAuthor":false,"prefix":"","firstName":"Ian","middleName":"","lastName":"Russell","suffix":""},{"id":338588497,"identity":"7c15649b-3179-4ff3-9a07-9363da6a3475","order_by":4,"name":"Danielle Nunnery","email":"","orcid":"","institution":"Appalachian State University","correspondingAuthor":false,"prefix":"","firstName":"Danielle","middleName":"","lastName":"Nunnery","suffix":""}],"badges":[],"createdAt":"2024-07-09 18:53:29","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4714037/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4714037/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":62309962,"identity":"98bca800-3d77-4838-b359-ce0106f020f7","added_by":"auto","created_at":"2024-08-12 19:55:23","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":745738,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003ePerformance Measurement Pressure and Relevant Themes Present in Clinic Manager Key Informant Interviews\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4714037/v1/a9a7d1bb24fd208d0ad850d9.png"},{"id":74935257,"identity":"2fc26202-1ece-490f-a940-046e2ca0825b","added_by":"auto","created_at":"2025-01-28 13:09:23","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1484216,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4714037/v1/2c72926c-4b52-4e8b-a272-71b45e537148.pdf"},{"id":62309961,"identity":"4b018627-3725-46fe-bf3a-ec49f5b03ee0","added_by":"auto","created_at":"2024-08-12 19:55:23","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":20171,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalFile1Table1SelectedCodedExtracts.docx","url":"https://assets-eu.researchsquare.com/files/rs-4714037/v1/79cba752f143b6087ee7abba.docx"},{"id":62310416,"identity":"17020271-c0bf-496b-9798-94f7da0f945b","added_by":"auto","created_at":"2024-08-12 20:03:23","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":18491,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalFile2ChecklistforQualitativeResearch.docx","url":"https://assets-eu.researchsquare.com/files/rs-4714037/v1/4704078dce5c5e02eea82744.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Perspectives on Performance Measurement Pressure and Telehealth Utilization in Free and Charitable Clinics: A Qualitative Interview Study","fulltext":[{"header":"Contributions to the Literature","content":"\u003cul\u003e\n \u003cli\u003eThis study fills a gap in the literature by exploring the impact of performance measurement pressure on telehealth utilization in free and charitable clinics (FCCs). \u0026nbsp; \u0026nbsp;\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eUsing the Consolidated Framework for Implementation Research (CFIR), it provides novel insights into unique barriers faced by FCCs. \u0026nbsp; \u0026nbsp;\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eThe qualitative interviews offer empirical evidence from free and charitable clinic leaders, enhancing the depth of understanding regarding challenges in this setting. \u0026nbsp; \u0026nbsp;\u003c/li\u003e\n \u003cli\u003eThe findings highlight the need for policymakers to revise performance metrics to support telehealth integration in FCCs, addressing the dual pressures of administrative burden and resource allocation in these essential healthcare providers.\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"Introduction","content":"\u003cp\u003eAlthough free and charitable clinics (FCCs) generally operate outside of traditional fee for service and capitated billing structures, these organizations typically coordinate, code, and document health services within or above industry standards (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). Specifically, free and charitable clinics participate in a national quality measures reporting program, similar to those utilized across other ambulatory settings. This study explores perspectives regarding performance measurement pressure as well as data collection and reporting requirements in FCCs as a barrier to the utilization of telehealth services due to the inherent resource needs associated with capturing data in person for many performance metrics. This pressure often stems from the need to demonstrate efficiency, effectiveness to stakeholders and comply with regulations. However, the current health system\u0026rsquo;s reliance on in-person data collection may discourage clinics from fully embracing telehealth solutions, as remote consultations may not easily align with or support meeting traditional quality metrics (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Consequently, this limitation in addition to other factors such as federal and state policies and patient preferences (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) could impede the integration of telehealth into clinic operations, potentially hindering access to care for underserved populations who could benefit from its convenience and accessibility.\u003c/p\u003e\n\u003ch3\u003ePerformance Measurement Pressure Defined\u003c/h3\u003e\n\u003cp\u003eThe Consolidated Framework for Implementation Research (CFIR) contains several evidence-based constructs designed to assist implementation science researchers with understanding, characterizing, and contextualizing factors impacting the implementation of new processes and technologies. One of those constructs, performance measurement pressure is defined as \u0026ldquo;Quality or benchmarking metrics or established service goals that drive implementation and/or delivery of the innovation\u0026rdquo; (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). This construct refers specifically to the impact of outside requirements including data reporting and evaluation on the feasibility, choice to implement, and success of an implementation (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAccording to the CFIR, performance measurement pressure can impact an implementation and an organization\u0026rsquo;s operations if the \u0026ldquo;the threat or reality of public reporting may motivate Inner Settings, especially late-adopters, to implement an innovation in an effort not to look bad compared to their competitors''. However, in some cases \u0026ldquo;public reporting can also have a negative influence if there is an adversarial relationship between the reporting entity and the Inner Setting\u0026rdquo; (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). In the case of this study, the inner setting studied includes a network of independently operated free and charitable clinics working to implement and utilize telehealth services.\u003c/p\u003e\n\u003ch3\u003eFree and Charitable Clinics (FCCs) in the United States\u003c/h3\u003e\n\u003cp\u003eFree and charitable clinics are essential healthcare providers offering medical services to uninsured and underinsured populations in the United States. Unlike traditional healthcare facilities, these clinics do not operate under fee-for-service or capitated billing structures. Instead, they rely on donations, grants, and volunteer support to function. These clinics provide a wide range of services, including primary care, dental care, mental health services, and sometimes specialty care, often at no or minimal cost to patients (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). Many clinics benefit from in-kind donations such as medical supplies, equipment, and professional volunteer services, which significantly reduce operational costs (6).\u003c/p\u003e \u003cp\u003eDespite limited financial resources, FCCs strive to maintain high standards of care and often participate in national quality measures reporting programs. This commitment to quality is crucial for demonstrating their value to funders and ensuring continued support. The operational model of these clinics emphasizes community-based care, targeting underserved populations who might otherwise lack access to essential health services (7).\u003c/p\u003e\n\u003ch3\u003eExploring Performance Measurement in the Free and Charitable Clinic Environment\u003c/h3\u003e\n\u003cp\u003ePerformance measurement in healthcare is a critical component for ensuring the delivery of high-quality care and improving patient outcomes. In recent years, the emphasis on performance metrics has extended beyond traditional healthcare settings to include FCCs, which serve as vital safety nets for underserved populations. According to the National Association of Free and Charitable Clinics (NAFC), these clinics provided nearly 6\u0026nbsp;million patient visits in 2022 (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). FCCs often operate with constrained budgets, relying heavily on volunteers and donations, which can complicate efforts to implement and maintain rigorous performance measurement systems. The pressure to measure and report performance can stem from multiple sources, including funding agencies, accreditation bodies, and internal organizational goals. Quality metrics, such as patient satisfaction, clinical outcomes, and operational efficiency, are crucial for demonstrating value and securing ongoing support (8). However, the resource-intensive nature of performance measurement can place significant strain on clinics that are already operating at capacity.\u003c/p\u003e \u003cp\u003eThis study explores perspectives regarding performance measurement pressure in FCCs as a barrier to the utilization of telehealth services and summarizes the challenges unique to this particular setting. The FCC environment was originally selected for this research study due to these unique constraints and the potential opportunity to explore key factors influencing the implementation and utilization of telehealth in a setting not dependent on traditional funding sources. Recently, not-for-profit organizations relying on funding from grants and donations have undergone significant scrutiny regarding the amount of their funds used for operations compared to the amount used for overhead costs (9). Performance measurement as a way to hold organizations accountable for the quality of services provided has been a concept throughout healthcare for quite some time (10), but this concept has recently gained focus with the sudden shift to telehealth placing emphasis on patient reported outcomes and introducing limiting factors such as time (11). FCCs not only contend with these complex needs to demonstrate the need for continued funding but also embrace the complex needs of the communities they serve. Clinics monitor outcomes for chronic conditions such as diabetes, hypertension, asthma, etc. and track indicators such as HbA1c levels in diabetic patients, blood pressure readings, and asthma control, the rate of vaccinations, cancer screenings (such as mammograms, colonoscopies, etc.), and other preventive services provided to patients.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eIn the spring of 2023, an interdisciplinary research team completed 13 key informant interviews with FCC leaders as part of a broader effort to utilize the Consolidated Framework on Implementation Research (CFIR) understand and contextualize the experience of implementing telehealth services during the height of the COVID-19 pandemic (12). The qualitative study exploring perspectives of clinic managers on telehealth utilization was conducted to supplement and provide context to quantitative data collected during a broader research effort to characterize telehealth implementation and utilization in FCCs (13). Qualitative research methods, particularly in-depth interviews, are well-suited to explore the nuanced experiences of clinic staff and administrators regarding performance measurement. Previous studies have highlighted the importance of contextual factors in performance measurement, suggesting that a one-size-fits-all approach may not be feasible (14).\u003c/p\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eRecruitment and Sample\u003c/h2\u003e \u003cp\u003e FCC clinic directors in one state were invited to complete the initial survey. Upon completing the survey, participants were invited to participate in key informant interviews. 20 survey respondents indicated initial interest to participate. Ultimately, 12 clinic leaders and 1 administrator from a state professional association completed interviews. All participants provided informed consent and were compensated \u003cspan\u003e$\u003c/span\u003e99 for their time. The sample size of 13 interviews was considered sufficient given the minimum sample threshold of 12 interviews frequently discussed in the literature (15\u0026ndash;17). Theoretical saturation was achieved from both a \u0026ldquo;coded saturation\u0026rdquo; and \u0026ldquo;meaning saturation\u0026rdquo;(18) perspective over the course of the interviews.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eData Collection\u003c/h2\u003e \u003cp\u003eInterviews were conducted via Zoom in March and April 2023 and included 10 clinic managers from sites that implemented telehealth during the pandemic, 2 clinic managers from clinics that did not implement telehealth services due to various barriers discussed, and 1 representative from a state-level professional association. The interviews were recorded and the audio transcriptions, in the form of Web Video Text Tracks (WebVTT) files, were captured and uploaded into MaxQDA 22.0 qualitative analysis software.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eMaxQDA was used to clean and edit the transcripts, which were then coded and analyzed to identify common themes, elements, and attributions. The study employed thematic analysis and grounded theory to explore key themes based on the Consolidated Framework for Implementation Research (CFIR), aiming to understand telehealth implementation and use in free and charitable clinics during and after the COVID-19 pandemic. Fundamental attitudes and themes regarding telehealth implementation and utilization were elicited using a qualitative descriptive technique with all constructs, including performance measurement pressure, being coded according to the model definitions. The research team used the key tenets of the CFIR as the initial framework for coding and interpreting the interview transcripts within MaxQDA. A team of four researchers coded the interviews, reviewing for CFIR constructs, identifying other relevant key themes through open coding, and ensuring agreement on the coded segments. The coded data documented key themes related to the CFIR and created open categories, axial codes, and key themes regarding telehealth implementation and utilization explored during the key informant interviews.\u003c/p\u003e \u003cp\u003eThe coded segments analyzed in this study were part of a comprehensive effort to code all interviews using the CFIR framework. After the initial coding, these segments were reviewed to assess intercoder consistency (ICC) (19) on the main themes and to ensure a shared understanding and application of CFIR constructs for categorizing and interpreting the data. Utilizing a real-time polling system, three reviewers evaluated a sample of 20 quotes from the clinic leadership interviews, aligning each quote with the appropriate CFIR construct or theme as verified by the team. The research team utilized the \u003cem\u003eGuidance for publishing qualitative research in informatics\u003c/em\u003e checklist for result reporting (20) (Additional file 2).\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eAn unexpected theme emerged specific to the CFIR construct of performance measurement pressure from the key informant interviews. While the team anticipated a variety of barriers, facilitators, challenges, and other factors to be discussed in addition to the key contracts of the CFIR model, the high intensity of comments and concern regarding performance measurement obligations as a defining limitation for telehealth as a care modality was not anticipated. This theme of performance measurement as a primary concern was present across all (n\u0026thinsp;=\u0026thinsp;13, 100%) key informant interviews as a concern and key consideration impacting perspectives regarding implementation and long-term sustainable use of telehealth services. While some clinic managers reported performance measurement pressure as a primary barrier to the widespread implementation of telehealth services at the clinic site, the discussion primarily focused on how current performance measurement elements, such as quality measures, could only be achieved through the delivery of in-person clinical care visits.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;1 featured in Additional File 1 provides a selection of coded extracts from the key informant interviews that illustrate the prominent theme of performance measurement pressure as a primary concern for FCC leaders implementing telehealth services. These extracts are broken down into the following sub-themes or axial codes as shown below in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eAxial Codes and Key Themes Identified Within the CFIR Construct of Performance Measurement Pressure:\u003c/h2\u003e \u003cp\u003e \u003col\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003ePerformance measurement pressure and barriers meeting individual quality measure\u003cb\u003es\u003c/b\u003e due to the virtual nature of telehealth: These extracts specifically highlight performance measurement pressure as it relates to difficulties capturing, measuring, or reporting necessary data required to meet specific performance metrics such as management of hypertension or conducting a diabetic foot exam.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003ePerformance measurement pressure and barriers capturing data and documenting general quality outcomes across patient populations via telehealth: These coded extracts specifically highlight clinic managers\u0026rsquo; perspectives regarding general concerns regarding measuring clinic performance that are not specific to challenges surrounding a specific quality measure.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003cspan\u003e \u003cli\u003e \u003cp\u003eHigh quality healthcare services \u0026amp; rigorous standards for performance measurement as a priority in FCCs: These extracts showcase the perspectives of clinic managers regarding the culture of performance measurement and quality focus present in FCCs during the COVID-19 pandemic.\u003c/p\u003e \u003c/li\u003e \u003c/span\u003e \u003c/ol\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eWhile free and charitable clinics operate under different constraints compared to other not for profit and for profit healthcare providers (21), these organizations face significant financial and performance measurement pressure. The insights gained from this study have the potential to inform policy and practice, helping to develop tailored and supportive approaches to performance measurement in FCCs. By understanding the perspectives of those on the front lines, stakeholders can better design systems that enhance quality without overwhelming the limited resources of these essential healthcare providers. The study illustrates how the emphasis on quality metrics, which often necessitate in-person clinic visits for data collection, can inadvertently constrain the adoption and effective use of telehealth services. The findings reveal that while telehealth offers significant potential to enhance access to care, especially for underserved populations, the rigid requirements of certain performance metrics pose substantial challenges. Clinic managers reported that metrics focusing on patient satisfaction, clinical outcomes, and process adherence frequently demand physical presence, making it difficult to fully leverage the flexibility and reach of telehealth. This tension underscores a critical gap in current performance measurement frameworks, which are not always adaptable to the nuances of telehealth modalities.\u003c/p\u003e \u003cp\u003eMoreover, the study highlights that the resource limitations endemic to FCCs exacerbate these challenges. Managers expressed concerns about the additional administrative burden and potential resource reallocation required to meet performance measurement standards while trying to expand telehealth services. This dual pressure can lead to a paradox where efforts to enhance care accessibility through telehealth are hindered by the very mechanisms intended to ensure care quality. To support the sustainable integration of telehealth in FCCs, there is a need for more flexible and inclusive performance measurement approaches. Policymakers and accrediting bodies should consider revising quality metrics to accommodate telehealth's distinct capabilities and limitations. This may involve developing new metrics that capture the quality of telehealth interactions and outcomes or modifying existing metrics to better reflect the hybrid nature of modern healthcare delivery. Future research should focus on identifying specific performance indicators that align with telehealth services and exploring innovative data collection methods that do not rely solely on in-person visits. Longitudinal studies could provide deeper insights into how performance measurement pressures evolve as telehealth becomes more embedded in healthcare practice.\u003c/p\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eLimitations\u003c/h2\u003e \u003cp\u003eThis study faced several limitations typical of qualitative research involving key informant interviews. As is common in qualitative research, this study employed a small sample size, which may limit the generalizability of the results to all free and charitable clinics (17). The potential for self-report and social desirability bias is high, given the lack of anonymity during Zoom interviews. Additionally, the unique circumstances of the COVID-19 pandemic created an atypical implementation environment for FCCs, which may not be generalizable to other time periods or settings. Another significant limitation is response bias. With only 12 out of 66 eligible clinic sites participating, the study might not capture the perspectives of those who did not prioritize telehealth or lacked the time to engage in the research (22).\u003c/p\u003e \u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study underscores the importance of developing and revising performance measurement frameworks to better support the integration of telehealth in FCCs. By aligning quality metrics with the realities of telehealth, stakeholders can ensure that these clinics can fully exploit the potential of telehealth to improve care access and quality for underserved populations. Addressing these challenges is crucial for the continued evolution and effectiveness of healthcare delivery in resource-constrained settings.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eFCC(s) - Free and Charitable Clinic(s)\u003c/p\u003e\n\u003cp\u003eCFIR - Consolidated Framework for Implementation Research\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was approved by Appalachian State University\u0026rsquo;s Institutional Review Board (Study # HS-22-39). Informed consent was obtained from all interviewed in this project. All participants provided consent for publication. \u0026nbsp;The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The research team would like to acknowledge the participation and support of the North Carolina Association of Free and Charitable Clinics (NCAFCC) and the NCAFCC member clinics and leadership who participated in the key informant interviews. \u0026nbsp;Their candor, expertise, and openness made this manuscript possible. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe author(s) received support for the research and publication of this article through a generous grant from the Appalachian Institute for Health and Wellness, formerly known as the Blue Cross North Carolina Institute for Health and Human Services (IHHS). The Implementation and Use of Telehealth by North Carolina Free and Charitable Clinics in the COVID 19 Era project was an interdisciplinary research project funded through a 2022-2023 Health Interdisciplinary Research Support Allocations (HISA) grant.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Information\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAuthors and Affiliations\u003c/p\u003e\n\u003cp\u003eAppalachian State University, Beaver College of Health Science, Department of Nutrition and Healthcare Management, 1179 State Farm Rd, Boone, NC 28607\u003c/p\u003e\n\u003cp\u003eAshley Parks, Andrew Wear, Julie Sakowski, Ian Russell, Danielle Nunnery\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAuthors represent an interdisciplinary collaboration within the Appalachian State University Beaver College of Health Sciences. The lead author, Ashley Parks, directed the project, led the key informant interviews of clinic managers with the active participation of all members of the interdisciplinary research team and leading the coding and analysis. \u0026nbsp; Ashley Parks, Julie Sakowski, and Danielle Nunnery obtained funding for this effort through an internal interdisciplinary grant and created the initial resource proposal, Institutional Review Board documentation, and recruitment documentation. Andrew Wear, Julie Sakowski, and Danielle Nunnery worked with Dr. Parks to create an interview guide de novo and pilot test the interview guide prior to the interviews. \u0026nbsp;Ashley Parks, Andrew Wear, Danielle Nunnery, and Ian Russell participated in the coding and cleaning of interview transcripts with Ashley Parks, Danielle Nunnery, Julie Sakowski, and Andrew Wear participating in coding validation through iterative comparison and polling exercises. Ashley Parks created the figure and table shared based on the team\u0026rsquo;s collective analysis. \u0026nbsp;Each team member brought unique perspectives and expertise to the study, enriching the data collection process. All authors were involved in conducting interviews, ensuring a comprehensive and nuanced understanding of the participants\u0026apos; perspectives. All authors have participating in the writing and review of this manuscript and approved the publication.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCorresponding Author\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eCorrespondence to Ashley Parks at [email protected]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was approved by Appalachian State University\u0026rsquo;s Institutional Review Board (Study # HS-22-39). Informed consent was obtained from all interviewed in this project.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll participants gave informed consent for their de-identified data to be published.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe coded extracts featured in the article are included in Table 1. \u0026nbsp;Additional data supporting the findings of this study are available from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors have no competing interests to disclose. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eOur Impact | National Association of Free \u0026amp; Charitable Clinics [Internet]. [cited 2023 Sep 26]. Available from: https://nafcclinics.org/our-impact/\u003c/li\u003e\n\u003cli\u003eTabaeeian RA, Hajrahimi B, Khoshfetrat A. A systematic review of telemedicine systems use barriers: primary health care providers\u0026rsquo; perspective. J Sci Technol Policy Manag. 2022 Jan 1;15(3):610\u0026ndash;35.\u003c/li\u003e\n\u003cli\u003eLin CCC, Dievler A, Robbins C, Sripipatana A, Quinn M, Nair S. Telehealth In Health Centers: Key Adoption Factors, Barriers, And Opportunities. Health Aff (Millwood). 2018 Dec;37(12):1967\u0026ndash;74.\u003c/li\u003e\n\u003cli\u003eUpdated CFIR Constructs \u0026ndash; The Consolidated Framework for Implementation Research [Internet]. [cited 2024 Jun 3]. Available from: https://cfirguide.org/constructs/\u003c/li\u003e\n\u003cli\u003eDamschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022 Oct 29;17(1):75.\u003c/li\u003e\n\u003cli\u003eFunding | Bureau of Primary Health Care [Internet]. [cited 2024 Jun 3]. Available from: https://bphc.hrsa.gov/funding\u003c/li\u003e\n\u003cli\u003eDarnell JS. Free clinics in the United States: a nationwide survey. Arch Intern Med. 2010 Jun 14;170(11):946\u0026ndash;53.\u003c/li\u003e\n\u003cli\u003eBerenson RA, Kaye DR. Grading a physician\u0026rsquo;s value--the misapplication of performance measurement. N Engl J Med. 2013 Nov 28;369(22):2079\u0026ndash;81.\u003c/li\u003e\n\u003cli\u003eHarris EE, Neely DG, Parsons LM. Nonprofit Performance Measurement and Reporting: Looking Forward. J Gov Nonprofit Account. 2022 Dec 1;11(1):51\u0026ndash;8.\u003c/li\u003e\n\u003cli\u003eGrants NRC (US) P on PM and D for PHPP. HEALTH PERFORMANCE MEASUREMENT IN THE PUBLIC SECTOR. In: Measuring Health Performance in the Public Sector: A Summary of Two Reports [Internet]. National Academies Press (US); 1999 [cited 2024 Jun 3]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK224448/\u003c/li\u003e\n\u003cli\u003eTiem JV, Wirtz E, Suiter N, Heeren A, Fuhrmeister L, Fortney J, et al. The Implementation of Measurement-Based Care in the Context of Telemedicine: Qualitative Study. JMIR Ment Health. 2022 Nov 24;9(11):e41601.\u003c/li\u003e\n\u003cli\u003eParks A. Free and Charitable Clinic Perspectives on the Implementation and Utilization of Telehealth Services During the COVID-19 Pandemic - Ashley V. Parks, Julie A. Sakowski, Andrew G. Wear, Ian Russell, Danielle Nunnery, 2023 [Internet]. [cited 2023 Dec 19]. Available from: https://journals-sagepub-com.proxy006.nclive.org/doi/10.1177/21501319231213783\u003c/li\u003e\n\u003cli\u003eSakowski JA, Parks A, Nunnery D, Wear A. Free and Charitable Clinic Telehealth Adoption and Utilization During the COVID-19 Era: The North Carolina Experience. Telemed Rep. 2023;4(1):215\u0026ndash;26.\u003c/li\u003e\n\u003cli\u003eSmith PC, Mossialos E, Papanicolas I. Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects. Health Syst Health Wealth Soc Well- Assess Case Invest Health Syst. 2010 Jul 6;247\u0026ndash;80.\u003c/li\u003e\n\u003cli\u003eBraun V, Clarke V. (Mis)conceptualising themes, thematic analysis, and other problems with Fugard and Potts\u0026rsquo; (2015) sample-size tool for thematic analysis. Int J Soc Res Methodol. 2016 Nov 1;19(6):739\u0026ndash;43.\u003c/li\u003e\n\u003cli\u003eFugard AJB, Potts HWW. Supporting thinking on sample sizes for thematic analyses: a quantitative tool. Int J Soc Res Methodol. 2015 Nov 2;18(6):669\u0026ndash;84.\u003c/li\u003e\n\u003cli\u003eVasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. 2018 Nov 21;18(1):148.\u003c/li\u003e\n\u003cli\u003eAldiabat K, Navenec CLL. Data Saturation: The Mysterious Step In Grounded Theory Method. Qual Rep. 2018 Jan 26;23(1):245\u0026ndash;61.\u003c/li\u003e\n\u003cli\u003eO\u0026rsquo;Connor C, Joffe H. Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines. Int J Qual Methods. 2020 Jan 1;19:1609406919899220.\u003c/li\u003e\n\u003cli\u003eAncker JS, Benda NC, Reddy M, Unertl KM, Veinot T. Guidance for publishing qualitative research in informatics. J Am Med Inform Assoc JAMIA. 2021 Sep 19;28(12):2743\u0026ndash;8.\u003c/li\u003e\n\u003cli\u003eWMJ [Internet]. [cited 2024 Jun 3]. The Safety Net\u0026rsquo;s Safety Net: Understanding the Crucial Role of Free Clinics in Cardiovascular Care. Available from: https://wmjonline.org/123no1/zellmer/\u003c/li\u003e\n\u003cli\u003eLeung L. Validity, reliability, and generalizability in qualitative research. J Fam Med Prim Care. 2015 Sep;4(3):324.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-4714037/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4714037/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch4\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/h4\u003e\n\u003cp\u003eFree and charitable clinics, essential for providing healthcare to uninsured and underinsured populations, face significant performance measurement pressures. These pressures, derived from the need to demonstrate quality and efficiency, may hinder the adoption of telehealth services due to a dearth of resources to meet the demands met by in-person data collection. This study explores manager perspectives on how performance measurement pressure impacts the implementation and sustainability of telehealth.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAn interdisciplinary research team conducted 13 key informant interviews with leaders from free and charitable clinics. The interviews, part of a larger research effort utilizing the Consolidated Framework for Implementation Research, aimed to understand free and charitable clinic implementation of telehealth during the COVID-19 pandemic. Interviews were recorded, transcribed, and analyzed using MaxQDA 22.0. Fundamental attitudes and themes regarding telehealth implementation and utilization were elicited using a qualitative descriptive technique with all constructs, including performance measurement pressure, being coded according to model definitions. The research team utilized the Guidance for publishing qualitative research in informatics checklist for result reporting.\u003c/p\u003e\n\u003ch4\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/h4\u003e\n\u003cp\u003eAn unexpected but prominent theme emerged: performance measurement pressure may be a limitation to wide scale telehealth implementation. All 13 interviewees highlighted this pressure as a notable concern. Performance metrics, often requiring in-person visits, were seen as barriers to the widespread adoption of telehealth. Some managers viewed these metrics as necessary for securing funding and ensuring quality but also reported that the current metrics were incompatible with telehealth, which posed challenges for sustainable telehealth integration.\u003c/p\u003e\n\u003ch4\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e\u003c/h4\u003e\n\u003cp\u003eThe study reveals that while telehealth could enhance care access for underserved populations, existing performance measurement frameworks hinder its adoption in free and charitable clinics. To support sustainable telehealth integration, there is a need for flexible performance metrics that accommodate telehealth’s unique capabilities. Findings from this study suggest that policymakers and accrediting bodies should consider revising quality metrics to better accommodate telehealth modalities. Future research should focus on developing specific telehealth performance indicators and data collection methods that do not rely solely on in-person visits. Addressing these issues is crucial for improving healthcare access and quality in resource-constrained settings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTrial registration\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e","manuscriptTitle":"Perspectives on Performance Measurement Pressure and Telehealth Utilization in Free and Charitable Clinics: A Qualitative Interview Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-12 19:55:18","doi":"10.21203/rs.3.rs-4714037/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"c89ecd5c-a3b9-442d-8e42-c337a3f76c19","owner":[],"postedDate":"August 12th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-01-28T13:08:57+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-12 19:55:18","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4714037","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4714037","identity":"rs-4714037","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}