{"paper_id":"1db90c4d-cd5b-4006-96bf-b367b5dd145b","body_text":"A Practical and Prescriptive Framework for Appropriate Implementation and Review of Artificial Intelligence (FAIR-AI) in Healthcare | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article A Practical and Prescriptive Framework for Appropriate Implementation and Review of Artificial Intelligence (FAIR-AI) in Healthcare Brian J. Wells, Hieu M. Nguyen, Andrew McWilliams, Matt Pallini, and 10 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5975624/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 11 Aug, 2025 Read the published version in npj Digital Medicine → Version 1 posted 11 You are reading this latest preprint version Abstract Health systems face the challenge of balancing innovation and safety to responsibly implement artificial intelligence (AI) solutions. The rapid proliferation, growing complexity, ethical considerations, and rising demand for these tools require timely and efficient processes for rigorous evaluation and ongoing monitoring. Current AI evaluation frameworks often lack the practical guidance for health systems to address these challenges. To fill this gap, we developed a prescriptive evaluation and governance framework informed by a literature review, in-depth interviews with key stakeholders, including patients, and a multidisciplinary design workshop. The resulting framework provides health systems an outline of the resources, structures, criteria, and template documents to enable pre-implementation evaluation and post-implementation monitoring of AI solutions. Health systems will need to treat this or any alternative framework as a living document to maintain relevance and effectiveness as the AI landscape and regulations continue to evolve. Biological sciences/Computational biology and bioinformatics/Machine learning Biological sciences/Computational biology and bioinformatics/Predictive medicine Health sciences/Health care/Health services Health sciences/Health care/Medical ethics Health sciences/Health care Physical sciences/Mathematics and computing/Information technology Physical sciences/Mathematics and computing/Software Artificial Intelligence Framework Practice Guideline Healthcare Biomedical Ethics Costs and Benefits Safety Harm Risk Regulations Predictive Model Software Algorithms Clinical Decision Support Systems Clinical Informatics Decision Analysis Health Equity Data Science Figures Figure 1 Figure 2 Figure 3 Introduction The healthcare industry is at an inflection point as the use of artificial intelligence-based tools rapidly expands, driven by the enhanced capabilities of modern electronic health record (EHR) systems and the advancement in artificial intelligence (AI) methods. The latest advancements in AI offer tremendous potential to improve patient outcomes, enhance patient experience, and increase efficiencies. 1 However, if hasty deployment of AI solutions bypasses rigorous evaluation steps, AI may paradoxically produce untoward results, such as introducing or amplifying health inequities, creating wasteful care and causing harm to those intended to be helped. 2 While AI has been used for clinical decision support in medicine for almost 50 years, evaluating the initial computer-based knowledge systems was relatively straightforward. 3 As AI use cases in healthcare expand, appropriately evaluating and monitoring AI solutions has become increasingly challenging due to more complex and, at times, inherently opaque AI models and methods with massive data requirements. 4 These challenges, combined with the rapid pace with which technology is being introduced and the increasing interest in utilizing innovative technologies, highlight the need for health systems to adopt new approaches for AI governance. The approaches need to be consistent with the historically high standards healthcare has maintained for responsibly adopting new technology. The necessity for oversight in healthcare is reflected in numerous publications demonstrating the gravity of potential risks that are uniquely present when AI intersects with decisions of consequence. 5 – 7 To harness the benefits of AI while appropriately managing its risks, health systems need to implement intentional, practical AI governance strategies. Despite the recent hype and emerging ubiquity of AI solutions, standardized approaches to guide the pre-implementation review and post-implementation monitoring of AI solutions in healthcare settings are lacking. In the context of enterprise risk management, health systems seek to understand, quantify, and manage risk to all stakeholders, be that to patients, employees, or the organization. To effectively address the direct and indirect risks of implementing AI solutions, governance frameworks must be comprehensive, standardized, repeatable, and transparent. However, existing evaluation frameworks often fail to meet these criteria, as they tend to be overly theoretical, lack practical and actionable guidance, or focus too narrowly on specific aspects of risk. 8 – 11 Considering these limitations, our organization, a large health system spanning the southeast and midwestern U.S., set out to create a practical, comprehensive AI governance framework focused on responsible AI implementation. In addition to providing standardized steps and clear criteria, we outline the key structures and resources necessary for a health system to operationalize a robust AI review and monitoring program. This project, Framework for the Appropriate Implementation and Review of AI (FAIR-AI) in healthcare, was guided by three aims: (1) to incorporate best practice recommendations from existing frameworks, guidelines, and regulations; (2) to understand the expectations and needs for an AI framework from a diverse set of health system stakeholders including patients, providers, operational leaders, and AI developers; and (3) to leverage a multidisciplinary group to synthesize best practice guidance and align stakeholder needs into a practical framework. Results Best practices and key considerations As a first step to inform the construct of FAIR-AI, we conducted a scoping review to identify the best practices and key considerations related to responsibly deploying AI in healthcare, these are summarized in Table 1 . The results were organized into several themes including validation, usefulness, transparency, and equity. Table 1 Best practices and key considerations in implementation of artificial intelligence Theme Best practices and key considerations Corresponding FAIR-AI component Validity Choose appropriate metrics to assess model performance. 12 – 14 IDR Question 10–11 Evaluate whether the model achieves appropriate performance with consideration of the clinical context. 14 IDR Question 10–11 Conduct validation studies to assess the model’s applicability to real-world clinical practice. 16 , 17 IDR Question 11 Usefulness Assess the AI solution’s net benefit by weighing benefits and risks and considering workflows that mitigate risks. 17 , 20 – 22 LRS Question 1 IDR Question 2–4 Assess usefulness based on factors such as resource utilization, time savings, ease of use, workflow integration, end-user perception, alert characteristics (e.g., mode, timing, and targets), and unintended consequences. 9 , 21 , 23 IDR Question 4 Transparency and equity Disclose information about the data and methods used to create the AI system. 24 , 25 Intake process LRS Question 5–8 IDR Question 8, 9, 13, 15 Disclose which patient characteristic variables that have historically been used to discriminate are included in the model and present clear justification. 20 , 26 – 28 LRS Question 3 IDR Question 5 Assess model performance across key patient subgroups. 10 , 29 , 30 IDR Question 6 Assess whether the AI system is equally accessible to those who may benefit. 24 IDR Question 7 Provide end-users with explanations and insights about the AI system’s processes and its potential biases and errors. 31 LRS Question 8 Transparency requirements Abbreviations: LRS = low risk screening; IDR = in-depth review LRS and IDR questions are shown in Tables 2 and 3 respectively. Evaluation of model validation Numerous publications and guidelines such as TRIPOD and TRIPOD-AI have described the reporting necessary to properly evaluate a risk prediction model, regardless of the underlying statistical or machine learning method. 12 , 13 An important consideration in model validation is careful selection of performance metrics. 14 For classification problems, calibration and classification metrics such as positive and negative predictive values should be considered in tandem with discrimination metrics. For regression problems, besides Mean Square Error (MSE), other metrics such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) can also be examined. 15 It is important to establish a model’s real-world applicability through dedicated validation studies. 16 , 17 The strength of evidence supporting validation and minimum performance standards should align with the intended use case, its potential risks, and the likelihood of performance variability once deployed based on the analytic approach or data sources (Supplementary Fig. 1). 16 , 17 , 14 Applying these traditional standards to evaluate the validity of generative AI models is uniquely challenging and frequently not possible. While the literature in this area is nascent, evaluation should still be performed and may require qualitative metrics such as user feedback and expert reviews, which can provide insights into performance, risks, and usefulness. 18 , 19 Evaluation of usefulness Deploying and maintaining AI solutions in healthcare requires significant resources and carries the potential for both risk and benefits, making it essential to evaluate whether a tool delivers a net benefit to the organization, clinical team, and patients. 20 , 21 Decision analyses can quantify the expected value of medical decisions, but they often require detailed cost estimates and complex modeling. Formal net benefit calculations simplify this process by integrating the relative value of benefits versus harms into a single metric. 17 , 22 However, a lack of objective data, the specific context, or the nature of the solution may render these calculations impractical. In these cases, net benefit provides a construct to guide qualitative discussions among subject matter experts, helping to weigh benefits and risks while considering workflows that mitigate risks. Additionally, a thorough assessment of clinical utility may require an impact study to evaluate a solution’s effects on factors such as resource utilization, time savings, ease of use, workflow integration, end-user perception, alert characteristics (e.g., mode, timing, and targets), and unintended consequences. 9 , 21 , 23 Evaluation of transparency and equity Given the potential for ethical and equity risks when deploying AI solutions in healthcare, transparency should be present to the degree that it is possible across all levels of the design, development, evaluation, and implementation of AI solutions to ensure fairness and accountability ( https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf) . 24,25 Specifically due to the potential for AI to perpetuate biases that could result in over- or under-treatment of certain populations, there must be a clear and defensible justification for including predictor variables that have historically been associated with discrimination, such as those outlined in the PROGRESS-Plus framework: place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, social capital, and personal characteristics linked to discrimination (e.g., age, disability, sexual orientation). 20 , 26 – 28 This is particularly important when these variables may act as proxies for other, more meaningful determinants of health. It is equally important to evaluate for patterns of algorithmic bias by monitoring outcomes for discordance between patient subgroups, as well as ensuring equal access to the AI solution itself when applicable. 10 , 29 , 30 , 24 Once an AI solution is implemented, transparency for end-users becomes a critical element for building trust and confidence, as well as empowering users to play a role in vigilance for potential unintended consequences. To achieve this post-implementation transparency, end-users should have information readily available that explains an AI solution’s intended use, limitations and potential risks ( https://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles) . 31 Generative AI presents unique challenges in terms of transparency. For example, deep learning relies on vast numbers of parameters drawn from increasingly large datasets and may be inherently unexplainable. When transparency is lacking there should be a greater emphasis on human oversight and education on limitations and risks, and this is an area of ongoing research. 19 Stakeholder needs and priorities Several systematic reviews emphasize the importance of stakeholder engagement in the design and implementation of AI solutions in healthcare; however, this aspect is often overlooked in the existing frameworks. 32 , 33 To create a practical and useful framework for health systems, we borrowed from user-centric design principles to first assess stakeholders’ priorities for an AI framework and their criteria for evaluating its successful implementation. We interviewed stakeholders including health system leaders, AI developers, providers, and patients. Our findings were previously presented at the 17th Annual Conference on the Science of Dissemination and Implementation, hosted by AcademyHealth. 34 The stakeholders expressed multiple priorities for an AI framework, particularly the need for: (1) risk tolerance assessments to weigh the potential patient harms of an AI solution against expected benefits, (2) a human-in-the-loop of any medical decisions made using an AI solution, (3) consideration that available, rigorous evidence may be limited when reviewing new AI solutions, and (4) awareness that solutions may not have been developed on diverse patient populations or data similar to the population in which a use case is proposed. Interviewees also highlighted the importance of ensuring that AI solutions are matched to institutional priorities and conform to all relevant regulations. They noted regulations can pose unique challenges for large, multi-state health systems. While patient safety and outcomes were identified as paramount, stakeholders also detailed the need for an AI framework to evaluate the impact of potential solutions on health system employees. When evaluating the successful implementation and utilization of an AI framework, stakeholders were consistent in explaining that the review process must operate in a timely manner, provide clear guidelines for AI developers, and ensure fair and consistent review processes that are applicable for both internally and externally developed solutions. Multiple interviewees cited the challenges presented by the rapid pace of AI innovation, expressing concerns that an overly bureaucratic and time-consuming review process could hinder the health system’s ability to keep pace with the wider healthcare market. Similarly, multiple senior leaders and AI developers explained that a successful AI framework would both encourage internal innovation and streamline the implementation of AI solutions in a safe manner. Framework for the Appropriate Implementation and Review of AI (FAIR-AI) in healthcare The project team synthesized these stakeholder needs and best practices into a set of requirements for health systems seeking to implement AI responsibly. FAIR-AI provides a detailed outline of: (i) foundational health system requirements –artifacts, personnel, processes, and tools; (ii) inclusion and exclusion criteria that specifically detail which AI solutions ought to be evaluated by FAIR-AI, thus defining scope and ensuring accountability; (iii) review questions in the form of a low-risk screening checklist and an in-depth review that provides a comprehensive evaluation of risk and benefits across the areas of development, validation, performance, ethics and equity, usefulness, compliance and regulations; (iv) discrete risk categories that map to the review criteria and are assigned to each AI solution and its intended use case; (v) safe implementation plans including monitoring and transparency requirements; (vi) an AI Label that consolidates information in an understandable format. These core components of FAIR-AI are also displayed in Fig. 1 . Foundational requirements Implementing a responsible AI framework requires that health systems have certain foundational elements in place: (i) artifacts include a set of guiding principles for AI implementation and an AI ethics statement (examples are shown in Supplementary Table 1), both of which should be endorsed at the highest level of the organization; (ii) personnel including an individual (or a team) with data science training who are accountable for reviews; (iii) process for escalation to an AI governance committee that has decision making authority and combines areas including ethics, cyber security, compliance, human resources, legal, data governance, clinical oversight, and research; (iv) and an inventory tool that serves as a single source of truth catalog that enables accountability for review, monitoring, and transparency requirements. It is important to establish that AI governance does not take the place of traditional governance but rather is a complementary function tightly integrated with system strategy, financial goals, cyber security, and data governance. Additionally, while the overarching structure of an AI governance framework like FAIR-AI may remain consistent over time, the rapid pace of change in technology and regulations requires a process for regular review and updating by subject matter experts. Intake process Internal leaders who are driving or responsible for the deployment and use of an AI solution within the enterprise are designated as business owners. In this framework, we require the business owner of an AI solution to provide a set of descriptive items through an intake form including: (i) existing problem to solve; (ii) clearly outlined intended use case; (iii) expected benefits; (iv) risks including worst-case scenario(s); (v) published and unpublished information on development, validation, and performance; and (vi) FDA approvals, if applicable. Inclusion and exclusion criteria Based on the premise that enterprise risk management must cast a wide net to be aware of potential risks, the inclusion for FAIR-AI review starts with a broad, general definition of AI solutions, which intentionally also includes solutions that do not directly relate to clinical care. We adopted the definition of AI from Matheny et al., as “computer system(s) capable of activities normally associated with human cognitive effort”. 35 We then provide additional scope specificity by excluding three general areas of AI. First, simple scoring systems and rules-based tools for which an end-user can reasonably be expected to evaluate and take responsibility for performance. Second, any physical medical device that also incorporates AI into its function, as there are well-established FDA regulations in place to evaluate and monitor risks associated with these devices ( https://www.fda.gov/medical-devices/classify-your-medical-device/how-determine-if-your-product-medical-device ). Third, any AI solution being considered under an Institutional Review Board (IRB)-approved research protocol that includes informed consent for the use of AI when human subjects are involved. Inclusion and exclusion criteria like these will need to be adapted to a health system’s local context. Discrete risk categories Risk evaluation considers the magnitude and importance of adverse consequences from a decision; and in the case of FAIR-AI, the decision to implement a new AI solution. 36 As there are numerous approaches and nomenclatures to define risk, local consensus on a clear definition is a critical initial step for a health system. We aimed for simplicity in our risk definition and the number of risk categories to ensure interpretability by diverse stakeholders. Additionally, we opted to pursue a qualitative determination of risk and avoid a purely quantitative, composite risk score approach. The requisite data rarely exist to perform such risk calculations reliably, and composites of weighted scores have the potential to dilute important individual risk factors as well as the nuance of risk mitigation offered by the workflows surrounding AI solutions (for example, requiring a human review of AI output before an action is taken). Thus, FAIR-AI determines the magnitude and importance of potential adverse effects through consensus between subject matter experts from a data science team, the business leader requesting the AI solution, and ad hoc consultation when additional expertise is needed. In this exercise, the group leverages published data and expert opinion to outline hypothetical worst-case scenarios and the harms that could occur as an indirect or direct result of output from the proposed AI solution. The consensus determines if those harms are minor, or not minor; and if not minor, are they sufficiently mitigated by the related implementation workflow and monitoring plan. This risk framework is like that proposed by the International Medical Devise Regulators Forum ( https://www.imdrf.org/documents/software-medical-device-possible-framework-risk-categorization-and-corresponding-considerations ). It is important here to note that every AI solution should be reviewed within the context of its intended use case, which includes the surrounding implementation workflows. After the FAIR-AI review, which is described in detail in the next section, each AI solution is designated as low, moderate, or high risk according to the following definitions (Fig. 2 ): Low risk: Potential adverse effects are expected to be minor and should be apparent to the end-user and business owner. No ethical, equity, compliance, or regulatory concerns were identified during a low-risk screen. Moderate risk: Based on an in-depth review, one or more of the following are present: (1) potential adverse effects are not minor but are adequately addressed by workflows; (2) ethical, equity, compliance, or regulatory issues are suspected or present, but are appropriately mitigated. High risk: Based on an in-depth review, one or more of the following are present: (1) potential adverse effects are notable and could have a significant negative impact on patients, teammates, individuals, or the enterprise; (2) ethical, equity, compliance, or regulatory issues suspected or present, but not adequately addressed; (3) insufficient evidence exists to recommend proceeding with implementation. For our health system, all AI solutions designated as high risk are escalated to the AI Governance committee where they undergo a multidisciplinary discussion. The discussion results in one of three final designations: (i) proceed to implementation under high-risk conditions; (ii) proceed to a pilot or research study; or (iii) do not proceed, implementation would create an intolerable risk for the organization. Low risk screening and in-depth review pathways As prioritized by our stakeholders, a responsible AI framework should be nimble enough to allow quick but thorough reviews of AI solutions that have a low chance of causing any harm to an individual or the organization. To that end, FAIR-AI incorporates a 2-step process: an initial low-risk screening pathway and a subsequent in-depth review pathway for all solutions that do not pass through the low-risk screen. For an AI solution to be designated low-risk, it must pass all the low-risk screening questions (Table 2 ). Table 2 Low Risk Screening Questions Question No or N/A YES 1 Adverse effects Is there reasonable potential that the AI introduces adverse effects that may be more than minor for patients, employees, and/or individuals? • There should be adequate evidence of implementation in similar settings to properly determine the potential for minor adverse effects. Low risk Proceed to in-depth review 2 Trust Is there reasonable potential that the AI may negatively impact trust between provider (or health system) and patient(s)? Low risk High risk Proceed to in-depth review 3 AI features, equity screen Does the AI algorithm incorporate (or inappropriately exclude) characteristics a that have historically been used to discriminate? • ‘YES’, if the developer cannot or will not show supporting evidence and clear supporting rationale. Low risk Proceed to in-depth review 4 AI output, equity screen Is it possible the AI will lead to decisions that differ across characteristics a that have historically been used to discriminate? • 'YES' if the intended problem to solve is one where disparities exist (e.g., access to healthcare resources, health outcomes, job applications, etc.). Low risk Proceed to in-depth review 5 Vulnerability considerations b Does the AI implementation intersect with any of the following healthcare settings/functions/populations: • Beginning of life (pre, peri, neo-natal) • End of life (hospice, DNR/code status, palliative care, advance directives) • Consent for treatment/research • Capacity for decision making Low risk High risk Proceed to in-depth review 6 Decision support Is the solution intended to provide decision support for any of the following? a. Medical coding b. Medical billing c. Employment or human resources d. Diagnosis, treatment, or prevention of disease Low risk Proceed to in-depth review 7 Sensitive data Does the AI interface with data that may require special consideration? a. Recording individuals b. Facial recognition c. Fingerprints d. Genetic data e. Claims/payor data f. Other sensitive data Low risk Proceed to in-depth review 8 Output explainability Will it be difficult for the intended user to understand how the AI solution arrived at its output or recommendation? Low risk Proceed to in-depth review 9 Ease of monitoring Post implementation, does the AI solution require advanced expertise to adequately monitor for expected and unexpected risks and benefits? • A 'NO' answer indicates the risks, any adverse effects, and benefits must be able to be routinely tracked by the business owner. Low risk Proceed to in-depth review 10 Other concern(s) Does the reviewer have any other significant concerns about the AI not captured within the low-risk screen? Low risk High Risk Proceed to in-depth review Abbreviations: AI = Artificial Intelligence; N/A = Not Available a PROGRESS-Plus: place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, social capital, personal characteristics associated with discrimination. 28 b Vulnerability: The conditions determined by physical, social, economic, and environmental factors or processes which increase the susceptibility of an individual, a community, assets, or systems to the impacts of hazards” (World Health Organization) In-depth review Should answers to any of the screening questions suggest potential risks, the AI solution moves on to an in-depth review guided by the questions presented in Table 3 . The in-depth review involves closer scrutiny of the AI solution by the data scientist and business owner and mandates a higher burden of proof that the potential benefits of the solution outweigh the potential risks identified during the screening process. Table 3 In-depth Review Questions Question No Yes Uncertain 1.1 Software as a medical device (SaMD) : Has the FDA cleared or approved the AI as SaMD a ? • The b usiness owner is responsible for producing the FDA letter. Continue to 1.2 Continue to 1.3 n/a 1.2 SaMD : Could the software meet the FDA definition of software as a device? a. The AI acquires, processes, or analyzes a medical image or signal related to a patient’s health. If this statement is TRUE, answer \"YES\". b. The AI displays medical information about a patient, study or guideline. If this statement is TRUE, answer \"NO\". c. The AI provides recommendations to a health care professional about prevention, diagnosis, or treatment of a disease AND provides the basis for recommendations, so the health care professional is not relying solely on the AI output for decision making. If this statement is TRUE, answer \"NO\". Moderate risk Continue to 2 High risk Continue to 2 High risk Continue to 2 1.3 SaMD Is the FDA approval of the AI as SaMD for the intended use within the healthcare organization? • The vendor and/or business owner are responsible for providing the FDA confirmation letter and all supporting documentation or data to allow for this determination. High risk Moderate risk n/a 2 Potential for significant adverse effects : Potential adverse effects are notable and could have a significant negative impact on patients, teammates, individuals, or the enterprise? • There should be adequate evidence of implementation in other similar settings to support a ‘NO’ answer. • The business owner is responsible for identifying supporting documentation Moderate risk High risk High risk 3 Adverse effects and workflows : Potential adverse effects are not minor but are adequately addressed by workflows to mitigate/control the risk? High risk Moderate risk High risk 4 Net benefit : There is substantial evidence that supports the benefits outweigh the risks that are expected from AI implementation? • Evidence should include implementation in other similar settings to support a ‘YES’ answer. • The business owner is responsible for identifying supporting documentation. High risk Moderate risk High risk 5 AI features, equity in depth : If the AI uses features that include characteristics b historically used to discriminate, then adequate evidence is provided for how they influence the output in the context of the intended use? High risk Moderate risk High risk 6 AI output, equity in depth : Adequate evidence is provided that the AI solution performs well in all key subgroups? • E.g., a model appropriately ranks patients according to risk and does not systematically underestimate or overestimate risk. High risk Moderate risk High risk 7 Access, equity : Is the AI system equally accessible to those who may benefit? High risk Moderate risk High risk 8 Medical billing, coding, human resource : a. Is an output of the AI that is related to medical billing or medical coding made part of a patient’s permanent record or released to a third party without the intervention of a human? b. Does the AI rank or categorize applicants or teammates for an intended use that is HR related? Moderate risk High risk n/a 9 Privacy/transparency : a. Does the AI solution record an individual without their knowledge? b. Is the organization ethically obligated to provide an explicit explanation that AI is being used or need to consent that AI is being used, but that is not part of solution or workflow? (e.g., based on potential risk(s) or if no human is in the loop) c. Does the AI solution analyze personal data that may lead to profiling or categorizing of individuals (excluding risk scoring for clinical diagnosis or clinical workflow prioritization)? Moderate risk High risk n/a 10 Development and validity Transparent reporting of development and validation steps is available AND no concerns are identified when evaluated against contemporary published AI reporting standards. If this statement is TRUE, answer \"YES\". • Answer ‘NO’, if supporting evidence is insufficient. • Answer ‘NO’, if concerns are present regarding the general validity of the model. • Answer ‘NO’ if the AI solution's methods or outputs are \"blackbox\" c and the AI implementation creates the potential for anything more than minor adverse effects on patients, employees, or individuals. High risk Moderate risk n/a 11 External performance a. Substantive evidence of external performance exists to the level that a local validation is not required? OR b. The development and validation data/environment are expected to be so similar to the local data/environment that local confirmation is NOT necessary (e.g., radiology imaging)? High risk Moderate risk n/a 12 Human oversight Is the AI solution directly or indirectly tied to workflow(s) that automate an action, documentation, or patient communication without human review, which may adversely affect patient health outcomes? Moderate risk High risk n/a Carry forward low-risk screening questions that are high risk 13 Sensitive data Does the AI interface with data that may require special consideration? a. Recording individuals b. Facial recognition c. Fingerprints d. Genetic data e. Claims/payor data f. Other sensitive data n/a High risk High risk 14 Trust Is there reasonable potential that the AI may negatively impact trust between provider (or health system) and patient(s)? n/a High risk High risk 15 Vulnerability considerations d Does the AI implementation intersect with any of the following healthcare settings/functions/populations: • Beginning of life (pre, peri, neo-natal) • End of life (hospice, DNR/code status, palliative care, advance directives) • Consent for treatment/research • Capacity for decision making n/a High risk High risk 16 Other concern(s) Does the reviewer have any other significant concerns about the AI not captured within the low-risk screen? n/a High risk High risk Abbreviations: FDA = Food and Drug Administration; AI = Artificial Intelligence; n/a = not available a SaMD: Software as Medical Device ( https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software ) b PROGRESS-Plus: place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, social capital, personal characteristics associated with discrimination. 28 c \"Blackbox\" could mean either the information is proprietary and not shared or a deep learning model, which due to its complexity cannot be understood. d Vulnerability: The conditions determined by physical, social, economic, and environmental factors or processes which increase the susceptibility of an individual, a community, assets, or systems to the impacts of hazards” (World Health Organization) Risk categorization and recommendations After application of the low-risk screening questions, the in-depth review questions (if necessary), and completion of the AI Governance committee review (if necessary), the proposed solution is assigned a final risk category, and a FAIR-AI Summary Statement is completed (an example is presented in Supplementary Box 1). At this point, an AI solution may need to go through other traditional governance requirements like a cyber security review, financial approvals, etc. If the AI solution ultimately is designated to move forward with implementation, then the data science team and business owners collaboratively develop a Safe AI Plan as outlined below. Monitoring and transparency requirements – Safe AI Plan Implemented AI solutions need continuous monitoring as they may fail to adapt to new data or practice changes, which can lead to inaccurate results and increasing bias over time. 37 , 38 Similarly, when AI solutions are made readily available in workflows, it becomes easier for the solution to be used outside of its approved intended use case, which may change its inherent risk profile. For these reasons, FAIR-AI requires a monitoring plan for every deployed AI solution consisting of an attestation by the business owner at regular intervals. The attestation affirms that: (i) the deployment is still aligned with the approved use case; (ii) the underlying data and related workflows have not substantially changed; (iii) the AI solution is delivering the expected benefit(s); (iv) no unforeseen risks have been identified; and (v) there are no concerns noted related to new regulations. If the original FAIR-AI review identified specific risks, then the attestation also includes an approach to evaluate each risk along with metrics (if applicable). These evaluation metrics may range from repeating a standard model performance evaluation to obtaining periodic end-user feedback on accuracy (e.g., for a generative AI solution). In addition to monitoring, all solutions categorized as high risk also require an AI Label (Fig. 3 ) and end-user education at regular intervals. Finally, in situations where an end-user could potentially not be aware they are interacting with AI instead of a human, the business owner must design implementation workflows that create transparency for the end-user (e.g., an alert, disclaimer, or consent as applicable). Discussion Health systems are under growing pressure to adopt an increasingly wide array of AI solutions some of which have enormous potential to transform healthcare, but many also introduce complex potential risks. The FAIR-AI framework described in this paper offers a prescriptive, practical, and scalable approach for evaluating AI solutions for use in healthcare. We have distilled the approach into a concise set of questions that a data science team member can use to quickly triage AI solutions, triggering a more time-intensive, rigorous review only when necessary. This practical approach is necessary given the volume of new solutions released and as AI becomes more ubiquitous across healthcare. By establishing formal review criteria and a consistent risk assessment process, institutions can ensure well-documented, defensible recommendations. Ultimately, by implementing FAIR-AI or a similar framework, health systems can foster a culture that upholds high standards for both internally and vendor-developed AI solutions, protecting patients and the care team, while being an early adopter harnessing actual AI benefits. There are many challenges to implementing and maintaining the framework we have developed. Successful implementation requires support from institutional leadership, along with the allocation of resources to maintain documentation, manage new requests, and ensure proper monitoring. Team members tasked with screening requests must be empowered to reject requests for solutions that do not provide adequate documentation for a thorough review, otherwise, the process may become slow and inefficient as they search for information. In our early experience, we have found many AI solutions lack the evidence needed to support implementation and first require further research or pilot testing, which demands substantial resources from either the health system or the vendor. Generative AI solutions present significant challenges when they intersect with patient care, particularly around the difficulty in explaining how a tool functions, the opaque nature of the data used for training, the lack of standardized performance, the extensive manual effort required to review output, the need for infrastructure to obtain user-feedback, and mechanisms for reporting inaccuracies. An often overlooked but critical challenge to the responsible implementation of AI is the significant training required for both evaluators and end-users. Several recently published guidelines provide structured approaches for assessing the reliability and transparency of large language models in healthcare. We recognize the importance of these emerging frameworks and plan to expand our AI evaluation framework to incorporate relevant elements from them. However, integrating these considerations will take time, as adapting existing validation strategies for generative AI requires careful refinement to ensure a practical, efficient, and reproducible process that aligns with stakeholder needs. 39 , 40 At our organization, we plan to review and adapt FAIR-AI at least annually, due to the rapid changes in the field and regulatory environment. For example, AI tools themselves are being used increasingly to monitor other AI solutions for safety, and future iterations of FAIR-AI will need to account for this evolving area. As AI solutions become pervasive across most workflows, all teammates play a role in being vigilant with an awareness of AI’s inherent limitations, security risks, and ethical considerations. To address this need to democratize responsibility, we are developing accompanying education that will enhance our organization’s responsible AI culture. There are numerous limitations to our approach to evaluating AI solutions as described in this paper. Our evaluation and monitoring processes require a significant commitment of time and resources. Some health systems may choose to rely only on evaluations provided by other entities, which reduces the burden on the health system and speeds up the adoption of new AI tools; however, this may introduce inherent bias and conflicts of interest. While the screening and in-depth review questions provide a structured approach, they are not exhaustive, and the effectiveness of the framework depends on the diligence and expertise of the evaluators. Additionally, this framework will require that organizations make modifications to meet their needs and risk tolerance and to ensure alignment with local regulatory requirements. FAIR-AI provides a practical template for health systems to adopt a process for the rigorous evaluation and monitoring of AI solutions. The prescriptive framework guided by explicit criteria is intentionally designed for health systems to use at the speed and scale required in real-world settings. This framework will enable institutions to carefully balance the desire to adopt innovative solutions while maintaining the highest standards for patient and care team safety. Methods Best practices - scoping review For the scoping review, we employed a pragmatic approach, utilizing Google Scholar as the primary search engine to locate pertinent published frameworks and papers. Search terms included: framework, guideline, evaluation, monitoring, transparency, explainability, artificial intelligence, validation, informatics, clinical decision support, ethics, equity, regulatory, legal, usefulness, risks, benefits, implementation, deployment, predictive model, machine learning, clinical utility, health. Additionally, we incorporated institutional guidelines from the National Institute of Standards and Technology (NIST) and the U.S. Food and Drug Administration (FDA) and conducted citation tracking to identify influential works. Stakeholder needs - interviews From March to April 2024, we conducted semi-structured interviews with executive leadership (N = 3), senior risk, compliance, and legal leaders (N = 6), data developers (N = 4), providers (N = 5), and patients (N = 5) from across our health system. We utilized purposive sampling methods to ensure we obtained stakeholder feedback from the five user domains (e.g., executive leadership) that we felt would be most impacted by the implementation of an AI framework. 39 Interviewees were identified by members of the study team. An interview guide was collaboratively developed by the study team, which included physicians, faculty, and health system leaders with expertise in ethics, equity, data science, and care delivery. Each interview lasted approximately 30 minutes, was completed via telephone or videoconference, and was facilitated by a qualitatively trained faculty member (JK). Interviews were audio recorded and transcribed verbatim. Transcripts were analyzed using both inductive and deductive coding methodologies, with thematic analysis employed to identify and organize emergent themes in the data. Three members of the study team collaboratively developed the coding dictionary (BJW, JK, AM), with the qualitative lead (JK) independently coding all transcripts and bringing any questions back to the team for review. Expert consensus - design workshop We convened a half-day, in-person workshop to synthesize the best practices identified from the literature review, the priorities outlined by stakeholders, and the consensus recommendations from a diverse team of subject matter experts. Workshop participants included individuals with expertise in legal affairs, regulatory compliance, cyber security, ethics, clinical care, clinical informatics, data science, and research (N = 33). As the starting point for the workshop, the primary project team created a draft framework outline. This outline, along with background information, pertinent literature, and summaries of stakeholder needs, were shared with attendees for review prior to the meeting. Ethical considerations The study was approved by the Wake Forest University Health Sciences Institutional Review Board (IRB00109544). Declarations Acknowledgments The study was supported by the Duke Endowment under award number AWD00002292. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Duke Endowment. We would like to express our gratitude to Dr. Reid Blackman for his valuable feedback on the design of the framework and Sally Baek and Michael Johnson from Atrium Health for their critical support with organizing the design workshop. Data availability The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request. Interview data are not made publicly available to protect the confidentiality of the interviewees, including senior leader participants. Author contributions AM and BJW supervised the study. Members of the FAIR-AI Consortium contributed to the conception and design of the study. AB, SC, PC, AC, MG, TH, MI, JK, AK, AM, HMN, MP, YJT, and BJW performed acquisition, analysis, and interpretation of the data and drafted and revised the manuscript. Competing interests The authors declare no competing interests. FAIR-AI Consortium Oguz Akbilgic, PhD 9 , Katie Barr, MSN, RN 10 , Amy Bovi, MA 5 , Alicia Bowers, JD 11 , Rikki Caffrey, MA, MS 12 , Michael S. Carroll, PhD 4 , Shih-Hsiung Chou, PhD 4 , Matthew CiRullo, DO, MBA 13 , Patricia Corn, MSN, RN 7 , Audrey Cuison, MS 8 , Stephen M. Downs, MD, MS, 14 Mary Gagen, MBA 8 , Natalie Hardy, BA 5 , Timothy Hetherington, MS 4 , Jason Heuay, MS 4 , McKenzie Isreal, MPH 2 , Kristina Katzovitz, MD 15 , Eric Kirkendall, MD, MBI 16 , Justin Kramer, PhD, MAT 6 , Andrew Kuzma, PhD 5 , Elsie Lindgren, MBA, BSN, RN 17 , Lindsey Lonergan, JD 18 , Elissa McKinley, BS 19 , Andrew McWilliams, MD, MPH 3 , Hieu M. Nguyen, MS 2 , Nicholas M. Pajewski, PhD 1 , Matt Pallini, MS 4 , Laura Sak-Castellano, BS 20 , Erika Setliff, DNP, RN 21 , Yhenneko J. Taylor, PhD 2 , Brian J. Wells, MD, PhD 1 , Gabe Wright, JD 18 . 9 Department of Cardiovascular Medicine, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA. 10 Chief Nursing Informatics Officer, Advocate Health, Milwaukee, Wisconsin, USA. 11 Innovation and Commercialization, Advocate Health, Charlotte, North Carolina, USA. 12 Clinical Ethics, Advocate Health, Chicago, Illinois, USA. 13 Family Medicine, Atrium Health, Charlotte, North Carolina, USA. 14 Department of Pediatrics, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA. 15 Chief Medical Information Officer, Advocate Health, Oakbrook, Illinois, USA. 16 Chief Medical Information Officer & Chief Information Officer, Advocate Health, Winston-Salem, North Carolina, USA. 17 Patient Safety, Advocate Health, Milwaukee, Wisconsin, USA. 18 Office of the General Counsel, Atrium Health, Charlotte, North Carolina, USA. 19 Cybersecurity Governance, Risk and Compliance, Advocate Health, Milwaukee, Wisconsin, USA. 20 Audit Services and Enterprise Risk Management, Advocate Health, Oak Brook, Illinois, USA. 21 Virtual Critical Care (Southeast Region Critical Care), Atrium Health, Charlotte, North Carolina, USA. References Rajpurkar, P., Chen, E., Banerjee, O. & Topol, E. J. AI in health and medicine. Nat. Med. 28, 31–38 (2022). Dedehayir, O. & Steinert, M. The hype cycle model: A review and future directions. Technol. Forecast. Soc. Change 108, 28–41 (2016). Shortliffe, E. H. & Buchanan, B. G. A model of inexact reasoning in medicine. Math. Biosci. 23, 351–379 (1975). Thirunavukarasu, A. J. et al. Large language models in medicine. Nat. Med. 29, 1930–1940 (2023). Birkstedt, T., Minkkinen, M., Tandon, A. & Mäntymäki, M. AI governance: themes, knowledge gaps and future agendas. Internet Res. 33, 133–167 (2023). Taeihagh, A. Governance of artificial intelligence. Policy Soc. 40, 137–157 (2021). Mäntymäki, M., Minkkinen, M., Birkstedt, T. & Viljanen, M. Defining organizational AI governance. AI Ethics 2, 603–609 (2022). Bedoya, A. D. et al. A framework for the oversight and local deployment of safe and high-quality prediction models. J. Am. Med. Inform. Assoc. JAMIA 29, 1631–1636 (2022). Reddy, S. et al. Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. 28, e100444 (2021). Vollmer, S. et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 368, l6927 (2020). van der Vegt, A. H. et al. Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework. J. Am. Med. Inform. Assoc. JAMIA 30, 1503–1515 (2023). Collins, G. S. et al. TRIPOD + AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 385, e078378 (2024). Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 13, 1–10 (2015). Steyerberg, E. W. et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiol. Camb. Mass 21, 128 (2010). Botchkarev, A. A New Typology Design of Performance Metrics to Measure Errors in Machine Learning Regression Algorithms. Interdiscip. J. Inf. Knowl. Manag. 14, 045–076 (2019). Moons, K. G. M., Altman, D. G., Vergouwe, Y. & Royston, P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606 (2009). Moons, K. G. M. et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98, 691–698 (2012). Park, Y.-J. et al. Assessing the research landscape and clinical utility of large language models: a scoping review. BMC Med. Inform. Decis. Mak. 24, 72 (2024). Bandi, A., Adapa, P. V. S. R. & Kuchi, Y. E. V. P. K. The Power of Generative AI: A Review of Requirements, Models, Input–Output Formats, Evaluation Metrics, and Challenges. Future Internet 15, 260 (2023). Wiens, J. et al. Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25, 1337–1340 (2019). Scott, I., Carter, S. & Coiera, E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health Care Inform. 28, e100251 (2021). Kappen, T. H. et al. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn. Progn. Res. 2, 11 (2018). Osheroff, J. A. et al. A Roadmap for National Action on Clinical Decision Support. J. Am. Med. Inform. Assoc. JAMIA 14, 141 (2007). Blackman, R. Ethical Machines: Your Concise Guide to Totally Unbiased, Transparent, and Respectful AI . (Harvard Business Review Press, 2022). Dankwa-Mullan, I. et al. A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle. J. Health Care Poor Underserved 32, 300–317 (2021). Paulus, J. K. & Kent, D. M. Race and Ethnicity: A Part of the Equation for Personalized Clinical Decision Making? Circ. Cardiovasc. Qual. Outcomes 10, e003823 (2017). Paulus, J. K. & Kent, D. M. Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. NPJ Digit. Med. 3, 99 (2020). O’Neill, J. et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J. Clin. Epidemiol. 67, 56–64 (2014). Liu, X. et al. The medical algorithmic audit. Lancet Digit. Health 4, e384–e397 (2022). Vasey, B. et al. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 377, e070904 (2022). Zerilli, J., Bhatt, U. & Weller, A. How transparency modulates trust in artificial intelligence. Patterns 3, (2022). de Hond, A. A. H. et al. Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit. Med. 5, 2 (2022). Crossnohere, N. L., Elsaid, M., Paskett, J., Bose-Brill, S. & Bridges, J. F. P. Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks. J. Med. Internet Res. 24, e36823 (2022). Kramer, J. et al. Developing a Framework for the Review and Oversight of Artificial Intelligence at a Large Healthcare Enterprise: Assessing the Needs and Priorities of Senior Health System Leadership, Providers, and Community Stakeholders. in (AcademyHealth, 2024). Matheny, M. E., Whicher, D. & Thadaney Israni, S. Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. JAMA 323, 509–510 (2020). Fischhoff, B., Watson, S. R. & Hope, C. Defining risk. Policy Sci. 17, 123–139 (1984). Feng, J. et al. Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare. Npj Digit. Med. 5, 1–9 (2022). Lu, J. et al. Learning under Concept Drift: A Review. Preprint at http://arxiv.org/abs/2004.05785 (2020). Gallifant, J. et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat. Med. 31, 60–69 (2025). Tam, T. Y. C. et al. A framework for human evaluation of large language models in healthcare derived from literature review. Npj Digit. Med. 7, 1–20 (2024). Additional Declarations No competing interests reported. Supplementary Files Fairframeworkversion20250206SupplementaryInformation.docx Cite Share Download PDF Status: Published Journal Publication published 11 Aug, 2025 Read the published version in npj Digital Medicine → Version 1 posted Editorial decision: Revision requested 19 Mar, 2025 Reviews received at journal 11 Mar, 2025 Reviews received at journal 10 Mar, 2025 Reviews received at journal 05 Mar, 2025 Reviewers agreed at journal 19 Feb, 2025 Reviewers agreed at journal 18 Feb, 2025 Reviewers agreed at journal 17 Feb, 2025 Reviewers invited by journal 17 Feb, 2025 Editor assigned by journal 07 Feb, 2025 Submission checks completed at journal 07 Feb, 2025 First submitted to journal 06 Feb, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-5975624\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":false,\"archivedVersions\":[],\"articleType\":\"Article\",\"associatedPublications\":[],\"authors\":[{\"id\":427333263,\"identity\":\"ab110bf0-8549-4d65-af04-d0bdec87fbed\",\"order_by\":0,\"name\":\"Brian J. Wells\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Biostatistics and Data Science, Wake Forest University School of Medicine\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Brian\",\"middleName\":\"J.\",\"lastName\":\"Wells\",\"suffix\":\"\"},{\"id\":427333264,\"identity\":\"030bae73-27c0-4d02-bee2-ced85ed454e5\",\"order_by\":1,\"name\":\"Hieu M. Nguyen\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABH0lEQVRIiWNgGAWjYBACA2YEm/EBkOABI4YChgRitDAbANVBtRgY4NaCxGaTgBhNQIs5O/MxCcYdtYkbjvceq/j5456MwbWzBx/8MPiTx8De+/gFFi2WzWxpEoxnjiduOHMu7WZPQjGPwe28ZMMeA4NiBp7jZhbYHHaYx0yCse1Y4oYbOWY3eBISgFpyzCR4DAwSGyTS2AywauH/BtFy/41Z4R+IFvOff/Bq4WEDaqkB2sJjxgyzhRlqC/MDrFrYjC0S2w4YzzyTYywtk5bAI3kbxDAwTmzjOcaGLcQMzh9+eONjW51s3/Ezhh/f2CTY893OATIq5BL72duYP+AK6QSGwwwKB9BF2cAxhRPUMcg3YBHGbcsoGAWjYBSMJAAAAvllAH8dWjkAAAAASUVORK5CYII=\",\"orcid\":\"\",\"institution\":\"Center for Health System Sciences, Atrium Health\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Hieu\",\"middleName\":\"M.\",\"lastName\":\"Nguyen\",\"suffix\":\"\"},{\"id\":427333265,\"identity\":\"78d43f3d-41da-41ec-89f3-0438d42ec95b\",\"order_by\":2,\"name\":\"Andrew McWilliams\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Internal Medicine, Division of Hospital Medicine, Atrium Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Andrew\",\"middleName\":\"\",\"lastName\":\"McWilliams\",\"suffix\":\"\"},{\"id\":427333266,\"identity\":\"d80aae74-2ef9-4dfc-b17e-6a02c19cca4e\",\"order_by\":3,\"name\":\"Matt Pallini\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Advanced Analytics, Enterprise Data Science, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Matt\",\"middleName\":\"\",\"lastName\":\"Pallini\",\"suffix\":\"\"},{\"id\":427333267,\"identity\":\"1df8c81d-bdf8-4c71-8332-5222837d4267\",\"order_by\":4,\"name\":\"Amy Bovi\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Conflict, Ethics, and Influence Program, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Amy\",\"middleName\":\"\",\"lastName\":\"Bovi\",\"suffix\":\"\"},{\"id\":427333268,\"identity\":\"b038a53d-c3aa-4623-adb2-7a5b8e3f8c34\",\"order_by\":5,\"name\":\"Andrew Kuzma\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Conflict, Ethics, and Influence Program, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Andrew\",\"middleName\":\"\",\"lastName\":\"Kuzma\",\"suffix\":\"\"},{\"id\":427333269,\"identity\":\"6fdbfed5-041b-4b43-ad03-aed48758d31d\",\"order_by\":6,\"name\":\"Justin Kramer\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Department of Family and Community Medicine, Wake Forest University School of Medicine\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Justin\",\"middleName\":\"\",\"lastName\":\"Kramer\",\"suffix\":\"\"},{\"id\":427333270,\"identity\":\"6d5f0736-5eaa-495c-814a-8665602576ac\",\"order_by\":7,\"name\":\"Shih-Hsiung Chou\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Advanced Analytics, Enterprise Data Science, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Shih-Hsiung\",\"middleName\":\"\",\"lastName\":\"Chou\",\"suffix\":\"\"},{\"id\":427333271,\"identity\":\"2fe07c1b-3324-433a-ac43-ef8b2fd24a0f\",\"order_by\":8,\"name\":\"Timothy Hetherington\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Advanced Analytics, Enterprise Data Science, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Timothy\",\"middleName\":\"\",\"lastName\":\"Hetherington\",\"suffix\":\"\"},{\"id\":427333272,\"identity\":\"b2051ce9-86fe-49ef-bd15-633ae355feb1\",\"order_by\":9,\"name\":\"Patricia Corn\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Compliance and Integrity, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Patricia\",\"middleName\":\"\",\"lastName\":\"Corn\",\"suffix\":\"\"},{\"id\":427333273,\"identity\":\"1ece3041-586e-46fe-9db9-4ff299550abe\",\"order_by\":10,\"name\":\"Yhenneko J. Taylor\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Center for Health System Sciences, Atrium Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Yhenneko\",\"middleName\":\"J.\",\"lastName\":\"Taylor\",\"suffix\":\"\"},{\"id\":427333274,\"identity\":\"6aa9a610-5e59-4270-966a-a9c5c039c768\",\"order_by\":11,\"name\":\"Audrey Cuison\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Advanced Analytics, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Audrey\",\"middleName\":\"\",\"lastName\":\"Cuison\",\"suffix\":\"\"},{\"id\":427333275,\"identity\":\"8edb5469-07a9-4167-9a46-0646c23c69a1\",\"order_by\":12,\"name\":\"Mary Gagen\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Advanced Analytics, Advocate Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Mary\",\"middleName\":\"\",\"lastName\":\"Gagen\",\"suffix\":\"\"},{\"id\":427333276,\"identity\":\"3725ba6c-be2c-4a9c-808d-28bb3f008eca\",\"order_by\":13,\"name\":\"McKenzie Isreal\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Center for Health System Sciences, Atrium Health\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"McKenzie\",\"middleName\":\"\",\"lastName\":\"Isreal\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-02-06 17:53:12\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-5975624/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-5975624/v1\",\"draftVersion\":[],\"editorialEvents\":[{\"content\":\"https://doi.org/10.1038/s41746-025-01900-y\",\"type\":\"published\",\"date\":\"2025-08-11T15:58:05+00:00\"}],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":78751835,\"identity\":\"21655681-2952-4765-97d1-f9bdd7cfe1de\",\"added_by\":\"auto\",\"created_at\":\"2025-03-18 11:58:35\",\"extension\":\"png\",\"order_by\":1,\"title\":\"Figure 1\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":90673,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eCore components of FAIR-AI\\u003c/strong\\u003e\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"1.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-5975624/v1/9722c050d1b87d605e5befaf.png\"},{\"id\":78752338,\"identity\":\"0f6f828e-7df7-4e91-81f2-294aa63445f7\",\"added_by\":\"auto\",\"created_at\":\"2025-03-18 12:06:35\",\"extension\":\"png\",\"order_by\":2,\"title\":\"Figure 2\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":72006,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eRisk categories as determined by FAIR-AI evaluation and escalation to AI Governance\\u003c/strong\\u003e\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"2.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-5975624/v1/81d675d82199e66a07ad4028.png\"},{\"id\":78751827,\"identity\":\"9ec67a6b-aa31-48d4-b5eb-127a146be2e8\",\"added_by\":\"auto\",\"created_at\":\"2025-03-18 11:58:35\",\"extension\":\"png\",\"order_by\":3,\"title\":\"Figure 3\",\"display\":\"\",\"copyAsset\":false,\"role\":\"figure\",\"size\":65368,\"visible\":true,\"origin\":\"\",\"legend\":\"\\u003cp\\u003e\\u003cstrong\\u003eAI Transparency Label\\u003c/strong\\u003e\\u003c/p\\u003e\",\"description\":\"\",\"filename\":\"3.png\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-5975624/v1/743d13a7d66cff68a6e8c7bc.png\"},{\"id\":89310706,\"identity\":\"f0e00f51-8fba-451f-a308-a95e3b6a49af\",\"added_by\":\"auto\",\"created_at\":\"2025-08-18 16:09:59\",\"extension\":\"pdf\",\"order_by\":0,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":1606403,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"manuscript.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-5975624/v1/f7387a86-249b-4e16-8574-6b472b248f9f.pdf\"},{\"id\":78751832,\"identity\":\"ab3aa0ae-b895-4493-8747-b827ebcfb3a3\",\"added_by\":\"auto\",\"created_at\":\"2025-03-18 11:58:35\",\"extension\":\"docx\",\"order_by\":0,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"supplement\",\"size\":264029,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Fairframeworkversion20250206SupplementaryInformation.docx\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-5975624/v1/18832850acfbe6e58ea419db.docx\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"A Practical and Prescriptive Framework for Appropriate Implementation and Review of Artificial Intelligence (FAIR-AI) in Healthcare\",\"fulltext\":[{\"header\":\"Introduction\",\"content\":\"\\u003cp\\u003eThe healthcare industry is at an inflection point as the use of artificial intelligence-based tools rapidly expands, driven by the enhanced capabilities of modern electronic health record (EHR) systems and the advancement in artificial intelligence (AI) methods. The latest advancements in AI offer tremendous potential to improve patient outcomes, enhance patient experience, and increase efficiencies.\\u003csup\\u003e\\u003cspan citationid=\\\"CR1\\\" class=\\\"CitationRef\\\"\\u003e1\\u003c/span\\u003e\\u003c/sup\\u003e However, if hasty deployment of AI solutions bypasses rigorous evaluation steps, AI may paradoxically produce untoward results, such as introducing or amplifying health inequities, creating wasteful care and causing harm to those intended to be helped.\\u003csup\\u003e\\u003cspan citationid=\\\"CR2\\\" class=\\\"CitationRef\\\"\\u003e2\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eWhile AI has been used for clinical decision support in medicine for almost 50 years, evaluating the initial computer-based knowledge systems was relatively straightforward.\\u003csup\\u003e\\u003cspan citationid=\\\"CR3\\\" class=\\\"CitationRef\\\"\\u003e3\\u003c/span\\u003e\\u003c/sup\\u003e As AI use cases in healthcare expand, appropriately evaluating and monitoring AI solutions has become increasingly challenging due to more complex and, at times, inherently opaque AI models and methods with massive data requirements.\\u003csup\\u003e\\u003cspan citationid=\\\"CR4\\\" class=\\\"CitationRef\\\"\\u003e4\\u003c/span\\u003e\\u003c/sup\\u003e These challenges, combined with the rapid pace with which technology is being introduced and the increasing interest in utilizing innovative technologies, highlight the need for health systems to adopt new approaches for AI governance. The approaches need to be consistent with the historically high standards healthcare has maintained for responsibly adopting new technology.\\u003c/p\\u003e \\u003cp\\u003eThe necessity for oversight in healthcare is reflected in numerous publications demonstrating the gravity of potential risks that are uniquely present when AI intersects with decisions of consequence.\\u003csup\\u003e\\u003cspan additionalcitationids=\\\"CR6\\\" citationid=\\\"CR5\\\" class=\\\"CitationRef\\\"\\u003e5\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR7\\\" class=\\\"CitationRef\\\"\\u003e7\\u003c/span\\u003e\\u003c/sup\\u003e To harness the benefits of AI while appropriately managing its risks, health systems need to implement intentional, practical AI governance strategies. Despite the recent hype and emerging ubiquity of AI solutions, standardized approaches to guide the pre-implementation review and post-implementation monitoring of AI solutions in healthcare settings are lacking.\\u003c/p\\u003e \\u003cp\\u003eIn the context of enterprise risk management, health systems seek to understand, quantify, and manage risk to all stakeholders, be that to patients, employees, or the organization. To effectively address the direct and indirect risks of implementing AI solutions, governance frameworks must be comprehensive, standardized, repeatable, and transparent. However, existing evaluation frameworks often fail to meet these criteria, as they tend to be overly theoretical, lack practical and actionable guidance, or focus too narrowly on specific aspects of risk.\\u003csup\\u003e\\u003cspan additionalcitationids=\\\"CR9 CR10\\\" citationid=\\\"CR8\\\" class=\\\"CitationRef\\\"\\u003e8\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR11\\\" class=\\\"CitationRef\\\"\\u003e11\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eConsidering these limitations, our organization, a large health system spanning the southeast and midwestern U.S., set out to create a practical, comprehensive AI governance framework focused on responsible AI implementation. In addition to providing standardized steps and clear criteria, we outline the key structures and resources necessary for a health system to operationalize a robust AI review and monitoring program. This project, Framework for the Appropriate Implementation and Review of AI (FAIR-AI) in healthcare, was guided by three aims: (1) to incorporate best practice recommendations from existing frameworks, guidelines, and regulations; (2) to understand the expectations and needs for an AI framework from a diverse set of health system stakeholders including patients, providers, operational leaders, and AI developers; and (3) to leverage a multidisciplinary group to synthesize best practice guidance and align stakeholder needs into a practical framework.\\u003c/p\\u003e\"},{\"header\":\"Results\",\"content\":\"\\u003cdiv id=\\\"Sec3\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eBest practices and key considerations\\u003c/h2\\u003e \\u003cp\\u003eAs a first step to inform the construct of FAIR-AI, we conducted a scoping review to identify the best practices and key considerations related to responsibly deploying AI in healthcare, these are summarized in Table\\u0026nbsp;\\u003cspan refid=\\\"Tab1\\\" class=\\\"InternalRef\\\"\\u003e1\\u003c/span\\u003e. The results were organized into several themes including validation, usefulness, transparency, and equity.\\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"Yes\\\" id=\\\"Tab1\\\" border=\\\"1\\\"\\u003e \\u003ccaption language=\\\"En\\\"\\u003e \\u003cdiv class=\\\"CaptionNumber\\\"\\u003eTable 1\\u003c/div\\u003e \\u003cdiv class=\\\"CaptionContent\\\"\\u003e \\u003cp\\u003eBest practices and key considerations in implementation of artificial intelligence\\u003c/p\\u003e \\u003c/div\\u003e \\u003c/caption\\u003e \\u003ccolgroup cols=\\\"3\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eTheme\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eBest practices and key considerations\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eCorresponding FAIR-AI component\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eValidity\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eChoose appropriate metrics to assess model performance.\\u003csup\\u003e\\u003cspan additionalcitationids=\\\"CR13\\\" citationid=\\\"CR12\\\" class=\\\"CitationRef\\\"\\u003e12\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 10\\u0026ndash;11\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eEvaluate whether the model achieves appropriate performance with consideration of the clinical context.\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 10\\u0026ndash;11\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eConduct validation studies to assess the model\\u0026rsquo;s applicability to real-world clinical practice.\\u003csup\\u003e\\u003cspan citationid=\\\"CR16\\\" class=\\\"CitationRef\\\"\\u003e16\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 11\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eUsefulness\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eAssess the AI solution\\u0026rsquo;s net benefit by weighing benefits and risks and considering workflows that mitigate risks.\\u003csup\\u003e\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e,\\u003cspan additionalcitationids=\\\"CR21\\\" citationid=\\\"CR20\\\" class=\\\"CitationRef\\\"\\u003e20\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR22\\\" class=\\\"CitationRef\\\"\\u003e22\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLRS Question 1\\u003c/p\\u003e \\u003cp\\u003eIDR Question 2\\u0026ndash;4\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eAssess usefulness based on factors such as resource utilization, time savings, ease of use, workflow integration, end-user perception, alert characteristics (e.g., mode, timing, and targets), and unintended consequences.\\u003csup\\u003e\\u003cspan citationid=\\\"CR9\\\" class=\\\"CitationRef\\\"\\u003e9\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR21\\\" class=\\\"CitationRef\\\"\\u003e21\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR23\\\" class=\\\"CitationRef\\\"\\u003e23\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 4\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003eTransparency and equity\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eDisclose information about the data and methods used to create the AI system.\\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR25\\\" class=\\\"CitationRef\\\"\\u003e25\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIntake process\\u003c/p\\u003e \\u003cp\\u003eLRS Question 5\\u0026ndash;8\\u003c/p\\u003e \\u003cp\\u003eIDR Question 8, 9, 13, 15\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eDisclose which patient characteristic variables that have historically been used to discriminate are included in the model and present clear justification.\\u003csup\\u003e\\u003cspan citationid=\\\"CR20\\\" class=\\\"CitationRef\\\"\\u003e20\\u003c/span\\u003e,\\u003cspan additionalcitationids=\\\"CR27\\\" citationid=\\\"CR26\\\" class=\\\"CitationRef\\\"\\u003e26\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR28\\\" class=\\\"CitationRef\\\"\\u003e28\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLRS Question 3\\u003c/p\\u003e \\u003cp\\u003eIDR Question 5\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eAssess model performance across key patient subgroups.\\u003csup\\u003e\\u003cspan citationid=\\\"CR10\\\" class=\\\"CitationRef\\\"\\u003e10\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR29\\\" class=\\\"CitationRef\\\"\\u003e29\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR30\\\" class=\\\"CitationRef\\\"\\u003e30\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 6\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eAssess whether the AI system is equally accessible to those who may benefit.\\u003csup\\u003e\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eIDR Question 7\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eProvide end-users with explanations and insights about the AI system\\u0026rsquo;s processes and its potential biases and errors.\\u003csup\\u003e\\u003cspan citationid=\\\"CR31\\\" class=\\\"CitationRef\\\"\\u003e31\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLRS Question 8\\u003c/p\\u003e \\u003cp\\u003eTransparency requirements\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colspan=\\\"3\\\" nameend=\\\"c3\\\" namest=\\\"c1\\\"\\u003e \\u003cp\\u003eAbbreviations: LRS\\u0026thinsp;=\\u0026thinsp;low risk screening; IDR\\u0026thinsp;=\\u0026thinsp;in-depth review\\u003c/p\\u003e \\u003cp\\u003eLRS and IDR questions are shown in Tables\\u0026nbsp;\\u003cspan refid=\\\"Tab2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e and \\u003cspan refid=\\\"Tab3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e respectively.\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003c/div\\u003e\\n\\u003ch3\\u003eEvaluation of model validation\\u003c/h3\\u003e\\n\\u003cp\\u003eNumerous publications and guidelines such as TRIPOD and TRIPOD-AI have described the reporting necessary to properly evaluate a risk prediction model, regardless of the underlying statistical or machine learning method.\\u003csup\\u003e\\u003cspan citationid=\\\"CR12\\\" class=\\\"CitationRef\\\"\\u003e12\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR13\\\" class=\\\"CitationRef\\\"\\u003e13\\u003c/span\\u003e\\u003c/sup\\u003e An important consideration in model validation is careful selection of performance metrics.\\u003csup\\u003e\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e For classification problems, calibration and classification metrics such as positive and negative predictive values should be considered in tandem with discrimination metrics. For regression problems, besides Mean Square Error (MSE), other metrics such as Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) can also be examined.\\u003csup\\u003e\\u003cspan citationid=\\\"CR15\\\" class=\\\"CitationRef\\\"\\u003e15\\u003c/span\\u003e\\u003c/sup\\u003e It is important to establish a model\\u0026rsquo;s real-world applicability through dedicated validation studies.\\u003csup\\u003e\\u003cspan citationid=\\\"CR16\\\" class=\\\"CitationRef\\\"\\u003e16\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e\\u003c/sup\\u003e The strength of evidence supporting validation and minimum performance standards should align with the intended use case, its potential risks, and the likelihood of performance variability once deployed based on the analytic approach or data sources (Supplementary Fig.\\u0026nbsp;1).\\u003csup\\u003e\\u003cspan citationid=\\\"CR16\\\" class=\\\"CitationRef\\\"\\u003e16\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR14\\\" class=\\\"CitationRef\\\"\\u003e14\\u003c/span\\u003e\\u003c/sup\\u003e Applying these traditional standards to evaluate the validity of generative AI models is uniquely challenging and frequently not possible. While the literature in this area is nascent, evaluation should still be performed and may require qualitative metrics such as user feedback and expert reviews, which can provide insights into performance, risks, and usefulness.\\u003csup\\u003e\\u003cspan citationid=\\\"CR18\\\" class=\\\"CitationRef\\\"\\u003e18\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003eEvaluation of usefulness\\u003c/h3\\u003e\\n\\u003cp\\u003eDeploying and maintaining AI solutions in healthcare requires significant resources and carries the potential for both risk and benefits, making it essential to evaluate whether a tool delivers a net benefit to the organization, clinical team, and patients.\\u003csup\\u003e\\u003cspan citationid=\\\"CR20\\\" class=\\\"CitationRef\\\"\\u003e20\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR21\\\" class=\\\"CitationRef\\\"\\u003e21\\u003c/span\\u003e\\u003c/sup\\u003e Decision analyses can quantify the expected value of medical decisions, but they often require detailed cost estimates and complex modeling. Formal net benefit calculations simplify this process by integrating the relative value of benefits versus harms into a single metric.\\u003csup\\u003e\\u003cspan citationid=\\\"CR17\\\" class=\\\"CitationRef\\\"\\u003e17\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR22\\\" class=\\\"CitationRef\\\"\\u003e22\\u003c/span\\u003e\\u003c/sup\\u003e However, a lack of objective data, the specific context, or the nature of the solution may render these calculations impractical. In these cases, net benefit provides a construct to guide qualitative discussions among subject matter experts, helping to weigh benefits and risks while considering workflows that mitigate risks. Additionally, a thorough assessment of clinical utility may require an impact study to evaluate a solution\\u0026rsquo;s effects on factors such as resource utilization, time savings, ease of use, workflow integration, end-user perception, alert characteristics (e.g., mode, timing, and targets), and unintended consequences.\\u003csup\\u003e\\u003cspan citationid=\\\"CR9\\\" class=\\\"CitationRef\\\"\\u003e9\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR21\\\" class=\\\"CitationRef\\\"\\u003e21\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR23\\\" class=\\\"CitationRef\\\"\\u003e23\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003eEvaluation of transparency and equity\\u003c/h3\\u003e\\n\\u003cp\\u003eGiven the potential for ethical and equity risks when deploying AI solutions in healthcare, transparency should be present to the degree that it is possible across all levels of the design, development, evaluation, and implementation of AI solutions to ensure fairness and accountability (\\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf)\\u003c/span\\u003e\\u003cspan address=\\\"https://nvlpubs.nist.gov/nistpubs/ai/nist.ai.100-1.pdf)\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e.\\u003csup\\u003e24,25\\u003c/sup\\u003e Specifically due to the potential for AI to perpetuate biases that could result in over- or under-treatment of certain populations, there must be a clear and defensible justification for including predictor variables that have historically been associated with discrimination, such as those outlined in the PROGRESS-Plus framework: place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, social capital, and personal characteristics linked to discrimination (e.g., age, disability, sexual orientation).\\u003csup\\u003e\\u003cspan citationid=\\\"CR20\\\" class=\\\"CitationRef\\\"\\u003e20\\u003c/span\\u003e,\\u003cspan additionalcitationids=\\\"CR27\\\" citationid=\\\"CR26\\\" class=\\\"CitationRef\\\"\\u003e26\\u003c/span\\u003e\\u0026ndash;\\u003cspan citationid=\\\"CR28\\\" class=\\\"CitationRef\\\"\\u003e28\\u003c/span\\u003e\\u003c/sup\\u003e This is particularly important when these variables may act as proxies for other, more meaningful determinants of health. It is equally important to evaluate for patterns of algorithmic bias by monitoring outcomes for discordance between patient subgroups, as well as ensuring equal access to the AI solution itself when applicable.\\u003csup\\u003e\\u003cspan citationid=\\\"CR10\\\" class=\\\"CitationRef\\\"\\u003e10\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR29\\\" class=\\\"CitationRef\\\"\\u003e29\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR30\\\" class=\\\"CitationRef\\\"\\u003e30\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR24\\\" class=\\\"CitationRef\\\"\\u003e24\\u003c/span\\u003e\\u003c/sup\\u003e Once an AI solution is implemented, transparency for end-users becomes a critical element for building trust and confidence, as well as empowering users to play a role in vigilance for potential unintended consequences. To achieve this post-implementation transparency, end-users should have information readily available that explains an AI solution\\u0026rsquo;s intended use, limitations and potential risks (\\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles)\\u003c/span\\u003e\\u003cspan address=\\\"https://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles)\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e.\\u003csup\\u003e31\\u003c/sup\\u003e Generative AI presents unique challenges in terms of transparency. For example, deep learning relies on vast numbers of parameters drawn from increasingly large datasets and may be inherently unexplainable. When transparency is lacking there should be a greater emphasis on human oversight and education on limitations and risks, and this is an area of ongoing research.\\u003csup\\u003e\\u003cspan citationid=\\\"CR19\\\" class=\\\"CitationRef\\\"\\u003e19\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e\\n\\u003ch3\\u003eStakeholder needs and priorities\\u003c/h3\\u003e\\n\\u003cp\\u003eSeveral systematic reviews emphasize the importance of stakeholder engagement in the design and implementation of AI solutions in healthcare; however, this aspect is often overlooked in the existing frameworks.\\u003csup\\u003e\\u003cspan citationid=\\\"CR32\\\" class=\\\"CitationRef\\\"\\u003e32\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR33\\\" class=\\\"CitationRef\\\"\\u003e33\\u003c/span\\u003e\\u003c/sup\\u003e To create a practical and useful framework for health systems, we borrowed from user-centric design principles to first assess stakeholders\\u0026rsquo; priorities for an AI framework and their criteria for evaluating its successful implementation. We interviewed stakeholders including health system leaders, AI developers, providers, and patients. Our findings were previously presented at the 17th Annual Conference on the Science of Dissemination and Implementation, hosted by AcademyHealth.\\u003csup\\u003e\\u003cspan citationid=\\\"CR34\\\" class=\\\"CitationRef\\\"\\u003e34\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eThe stakeholders expressed multiple priorities for an AI framework, particularly the need for: (1) risk tolerance assessments to weigh the potential patient harms of an AI solution against expected benefits, (2) a human-in-the-loop of any medical decisions made using an AI solution, (3) consideration that available, rigorous evidence may be limited when reviewing new AI solutions, and (4) awareness that solutions may not have been developed on diverse patient populations or data similar to the population in which a use case is proposed. Interviewees also highlighted the importance of ensuring that AI solutions are matched to institutional priorities and conform to all relevant regulations. They noted regulations can pose unique challenges for large, multi-state health systems. While patient safety and outcomes were identified as paramount, stakeholders also detailed the need for an AI framework to evaluate the impact of potential solutions on health system employees.\\u003c/p\\u003e \\u003cp\\u003e When evaluating the successful implementation and utilization of an AI framework, stakeholders were consistent in explaining that the review process must operate in a timely manner, provide clear guidelines for AI developers, and ensure fair and consistent review processes that are applicable for both internally and externally developed solutions. Multiple interviewees cited the challenges presented by the rapid pace of AI innovation, expressing concerns that an overly bureaucratic and time-consuming review process could hinder the health system\\u0026rsquo;s ability to keep pace with the wider healthcare market. Similarly, multiple senior leaders and AI developers explained that a successful AI framework would both encourage internal innovation and streamline the implementation of AI solutions in a safe manner.\\u003c/p\\u003e \\u003cdiv id=\\\"Sec8\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eFramework for the Appropriate Implementation and Review of AI (FAIR-AI) in healthcare\\u003c/h2\\u003e \\u003cp\\u003eThe project team synthesized these stakeholder needs and best practices into a set of requirements for health systems seeking to implement AI responsibly. FAIR-AI provides a detailed outline of: (i) foundational health system requirements \\u0026ndash;artifacts, personnel, processes, and tools; (ii) inclusion and exclusion criteria that specifically detail which AI solutions ought to be evaluated by FAIR-AI, thus defining scope and ensuring accountability; (iii) review questions in the form of a low-risk screening checklist and an in-depth review that provides a comprehensive evaluation of risk and benefits across the areas of development, validation, performance, ethics and equity, usefulness, compliance and regulations; (iv) discrete risk categories that map to the review criteria and are assigned to each AI solution and its intended use case; (v) safe implementation plans including monitoring and transparency requirements; (vi) an AI Label that consolidates information in an understandable format. These core components of FAIR-AI are also displayed in Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig1\\\" class=\\\"InternalRef\\\"\\u003e1\\u003c/span\\u003e.\\u003c/p\\u003e \\u003cp\\u003e \\u003c/p\\u003e \\u003c/div\\u003e\\n\\u003ch3\\u003eFoundational requirements\\u003c/h3\\u003e\\n\\u003cp\\u003e Implementing a responsible AI framework requires that health systems have certain foundational elements in place: (i) artifacts include a set of guiding principles for AI implementation and an AI ethics statement (examples are shown in Supplementary Table\\u0026nbsp;1), both of which should be endorsed at the highest level of the organization; (ii) personnel including an individual (or a team) with data science training who are accountable for reviews; (iii) process for escalation to an AI governance committee that has decision making authority and combines areas including ethics, cyber security, compliance, human resources, legal, data governance, clinical oversight, and research; (iv) and an inventory tool that serves as a single source of truth catalog that enables accountability for review, monitoring, and transparency requirements. It is important to establish that AI governance does not take the place of traditional governance but rather is a complementary function tightly integrated with system strategy, financial goals, cyber security, and data governance. Additionally, while the overarching structure of an AI governance framework like FAIR-AI may remain consistent over time, the rapid pace of change in technology and regulations requires a process for regular review and updating by subject matter experts.\\u003c/p\\u003e\\n\\u003ch3\\u003eIntake process\\u003c/h3\\u003e\\n\\u003cp\\u003eInternal leaders who are driving or responsible for the deployment and use of an AI solution within the enterprise are designated as business owners. In this framework, we require the business owner of an AI solution to provide a set of descriptive items through an intake form including: (i) existing problem to solve; (ii) clearly outlined intended use case; (iii) expected benefits; (iv) risks including worst-case scenario(s); (v) published and unpublished information on development, validation, and performance; and (vi) FDA approvals, if applicable.\\u003c/p\\u003e \\u003cdiv id=\\\"Sec11\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eInclusion and exclusion criteria\\u003c/h2\\u003e \\u003cp\\u003eBased on the premise that enterprise risk management must cast a wide net to be aware of potential risks, the inclusion for FAIR-AI review starts with a broad, general definition of AI solutions, which intentionally also includes solutions that do not directly relate to clinical care. We adopted the definition of AI from Matheny et al., as \\u0026ldquo;computer system(s) capable of activities normally associated with human cognitive effort\\u0026rdquo;.\\u003csup\\u003e\\u003cspan citationid=\\\"CR35\\\" class=\\\"CitationRef\\\"\\u003e35\\u003c/span\\u003e\\u003c/sup\\u003e We then provide additional scope specificity by excluding three general areas of AI. First, simple scoring systems and rules-based tools for which an end-user can reasonably be expected to evaluate and take responsibility for performance. Second, any physical medical device that also incorporates AI into its function, as there are well-established FDA regulations in place to evaluate and monitor risks associated with these devices (\\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://www.fda.gov/medical-devices/classify-your-medical-device/how-determine-if-your-product-medical-device\\u003c/span\\u003e\\u003cspan address=\\\"https://www.fda.gov/medical-devices/classify-your-medical-device/how-determine-if-your-product-medical-device\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e). Third, any AI solution being considered under an Institutional Review Board (IRB)-approved research protocol that includes informed consent for the use of AI when human subjects are involved. Inclusion and exclusion criteria like these will need to be adapted to a health system\\u0026rsquo;s local context.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec12\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eDiscrete risk categories\\u003c/h2\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"BlockQuote\\\"\\u003e \\u003cp\\u003eRisk evaluation considers the magnitude and importance of adverse consequences from a decision; and in the case of FAIR-AI, the decision to implement a new AI solution.\\u003csup\\u003e\\u003cspan citationid=\\\"CR36\\\" class=\\\"CitationRef\\\"\\u003e36\\u003c/span\\u003e\\u003c/sup\\u003e As there are numerous approaches and nomenclatures to define risk, local consensus on a clear definition is a critical initial step for a health system. We aimed for simplicity in our risk definition and the number of risk categories to ensure interpretability by diverse stakeholders. Additionally, we opted to pursue a qualitative determination of risk and avoid a purely quantitative, composite risk score approach. The requisite data rarely exist to perform such risk calculations reliably, and composites of weighted scores have the potential to dilute important individual risk factors as well as the nuance of risk mitigation offered by the workflows surrounding AI solutions (for example, requiring a human review of AI output before an action is taken). Thus, FAIR-AI determines the magnitude and importance of potential adverse effects through consensus between subject matter experts from a data science team, the business leader requesting the AI solution, and ad hoc consultation when additional expertise is needed. In this exercise, the group leverages published data and expert opinion to outline hypothetical worst-case scenarios and the harms that could occur as an indirect or direct result of output from the proposed AI solution. The consensus determines if those harms are minor, or not minor; and if not minor, are they sufficiently mitigated by the related implementation workflow and monitoring plan. This risk framework is like that proposed by the International Medical Devise Regulators Forum (\\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://www.imdrf.org/documents/software-medical-device-possible-framework-risk-categorization-and-corresponding-considerations\\u003c/span\\u003e\\u003cspan address=\\\"https://www.imdrf.org/documents/software-medical-device-possible-framework-risk-categorization-and-corresponding-considerations\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e). It is important here to note that every AI solution should be reviewed within the context of its intended use case, which includes the surrounding implementation workflows.\\u003c/p\\u003e \\u003cp\\u003eAfter the FAIR-AI review, which is described in detail in the next section, each AI solution is designated as low, moderate, or high risk according to the following definitions (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e):\\u003c/p\\u003e \\u003c/div\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003cul\\u003e \\u003cli\\u003e \\u003cp\\u003eLow risk: Potential adverse effects are expected to be minor and should be apparent to the end-user and business owner. No ethical, equity, compliance, or regulatory concerns were identified during a low-risk screen.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eModerate risk: Based on an in-depth review, one or more of the following are present: (1) potential adverse effects are not minor but are adequately addressed by workflows; (2) ethical, equity, compliance, or regulatory issues are suspected or present, but are appropriately mitigated.\\u003c/p\\u003e \\u003c/li\\u003e \\u003cli\\u003e \\u003cp\\u003eHigh risk: Based on an in-depth review, one or more of the following are present: (1) potential adverse effects are notable and could have a significant negative impact on patients, teammates, individuals, or the enterprise; (2) ethical, equity, compliance, or regulatory issues suspected or present, but not adequately addressed; (3) insufficient evidence exists to recommend proceeding with implementation.\\u003c/p\\u003e \\u003c/li\\u003e \\u003c/ul\\u003e \\u003c/p\\u003e \\u003cp\\u003e \\u003c/p\\u003e \\u003cp\\u003eFor our health system, all AI solutions designated as high risk are escalated to the AI Governance committee where they undergo a multidisciplinary discussion. The discussion results in one of three final designations: (i) proceed to implementation under high-risk conditions; (ii) proceed to a pilot or research study; or (iii) do not proceed, implementation would create an intolerable risk for the organization.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec13\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eLow risk screening and in-depth review pathways\\u003c/h2\\u003e \\u003cp\\u003eAs prioritized by our stakeholders, a responsible AI framework should be nimble enough to allow quick but thorough reviews of AI solutions that have a low chance of causing any harm to an individual or the organization. To that end, FAIR-AI incorporates a 2-step process: an initial low-risk screening pathway and a subsequent in-depth review pathway for all solutions that do not pass through the low-risk screen. For an AI solution to be designated low-risk, it must pass all the low-risk screening questions (Table\\u0026nbsp;\\u003cspan refid=\\\"Tab2\\\" class=\\\"InternalRef\\\"\\u003e2\\u003c/span\\u003e).\\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"Yes\\\" id=\\\"Tab2\\\" border=\\\"1\\\"\\u003e \\u003ccaption language=\\\"En\\\"\\u003e \\u003cdiv class=\\\"CaptionNumber\\\"\\u003eTable 2\\u003c/div\\u003e \\u003cdiv class=\\\"CaptionContent\\\"\\u003e \\u003cp\\u003eLow Risk Screening Questions\\u003c/p\\u003e \\u003c/div\\u003e \\u003c/caption\\u003e \\u003ccolgroup cols=\\\"4\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eQuestion\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eNo or N/A\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eYES\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e1\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAdverse effects\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs there reasonable potential that the AI introduces adverse effects that may be \\u003cspan type=\\\"Underline\\\" class=\\\"Underline\\\" name=\\\"Emphasis\\\"\\u003emore than minor\\u003c/span\\u003e for patients, employees, and/or individuals?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThere should be adequate evidence of implementation in similar settings to properly determine the potential for minor adverse effects.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e2\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eTrust\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs there reasonable potential that the AI may negatively impact trust between provider (or health system) and patient(s)?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e3\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAI features, equity screen\\u003c/b\\u003e \\u003c/p\\u003e \\u003cp\\u003eDoes the AI algorithm incorporate (or inappropriately exclude) characteristics\\u003csup\\u003ea\\u003c/sup\\u003e that have historically been used to discriminate?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003e\\u0026lsquo;YES\\u0026rsquo;, if the developer cannot or will not show supporting evidence and clear supporting rationale.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e4\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAI output, equity screen\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs it possible the AI will lead to decisions that differ across characteristics\\u003csup\\u003ea\\u003c/sup\\u003e that have historically been used to discriminate?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003e'YES' if the intended problem to solve is one where disparities exist (e.g., access to healthcare resources, health outcomes, job applications, etc.).\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e5\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eVulnerability considerations\\u003c/b\\u003e\\u003csup\\u003e\\u003cb\\u003eb\\u003c/b\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the AI implementation intersect with any of the following healthcare settings/functions/populations:\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Beginning of life (pre, peri, neo-natal)\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; End of life (hospice, DNR/code status, palliative care, advance directives)\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Consent for treatment/research\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Capacity for decision making\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e6\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eDecision support\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs the solution intended to provide decision support for any of the following?\\u003c/p\\u003e \\u003cp\\u003ea. Medical coding\\u003c/p\\u003e \\u003cp\\u003eb. Medical billing\\u003c/p\\u003e \\u003cp\\u003ec. Employment or human resources\\u003c/p\\u003e \\u003cp\\u003ed. Diagnosis, treatment, or prevention of disease\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e7\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eSensitive data\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the AI interface with data that may require special consideration?\\u003c/p\\u003e \\u003cp\\u003ea. Recording individuals\\u003c/p\\u003e \\u003cp\\u003eb. Facial recognition\\u003c/p\\u003e \\u003cp\\u003ec. Fingerprints\\u003c/p\\u003e \\u003cp\\u003ed. Genetic data\\u003c/p\\u003e \\u003cp\\u003ee. Claims/payor data\\u003c/p\\u003e \\u003cp\\u003ef. Other sensitive\\u0026nbsp;data\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e8\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eOutput explainability\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eWill it be difficult for the intended user to understand how the AI solution arrived at its output or recommendation?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e9\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eEase of monitoring\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003ePost implementation, does the AI solution require advanced expertise to adequately monitor for expected and unexpected risks and benefits?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eA 'NO' answer indicates the risks, any adverse effects, and benefits must be able to be routinely tracked by the business owner.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e10\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eOther concern(s)\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the reviewer have any other significant concerns about the AI not captured within the low-risk screen?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eLow risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh Risk\\u003c/p\\u003e \\u003cp\\u003eProceed to in-depth review\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colspan=\\\"4\\\" nameend=\\\"c4\\\" namest=\\\"c1\\\"\\u003e \\u003cp\\u003eAbbreviations: AI\\u0026thinsp;=\\u0026thinsp;Artificial Intelligence; N/A\\u0026thinsp;=\\u0026thinsp;Not Available\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003ea\\u003c/sup\\u003ePROGRESS-Plus: place of residence,\\u0026nbsp;race/ethnicity/culture/language, occupation, gender/sex,\\u0026nbsp;religion,\\u0026nbsp;education, socioeconomic status, social\\u0026nbsp;capital, personal characteristics associated with discrimination.\\u003csup\\u003e\\u003cspan citationid=\\\"CR28\\\" class=\\\"CitationRef\\\"\\u003e28\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003eb\\u003c/sup\\u003eVulnerability: The\\u0026nbsp;conditions determined by physical, social, economic, and environmental factors or processes which increase the susceptibility of an individual, a community, assets, or systems to the impacts of hazards\\u0026rdquo; (World Health Organization)\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec14\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eIn-depth review\\u003c/h2\\u003e \\u003cp\\u003eShould answers to any of the screening questions suggest potential risks, the AI solution moves on to an in-depth review guided by the questions presented in Table\\u0026nbsp;\\u003cspan refid=\\\"Tab3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e. The in-depth review involves closer scrutiny of the AI solution by the data scientist and business owner and mandates a higher burden of proof that the potential benefits of the solution outweigh the potential risks identified during the screening process.\\u003c/p\\u003e \\u003cp\\u003e \\u003cdiv class=\\\"gridtable\\\"\\u003e\\u003ctable float=\\\"Yes\\\" id=\\\"Tab3\\\" border=\\\"1\\\"\\u003e \\u003ccaption language=\\\"En\\\"\\u003e \\u003cdiv class=\\\"CaptionNumber\\\"\\u003eTable 3\\u003c/div\\u003e \\u003cdiv class=\\\"CaptionContent\\\"\\u003e \\u003cp\\u003eIn-depth Review Questions\\u003c/p\\u003e \\u003c/div\\u003e \\u003c/caption\\u003e \\u003ccolgroup cols=\\\"5\\\"\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c1\\\" colnum=\\\"1\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c2\\\" colnum=\\\"2\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c3\\\" colnum=\\\"3\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c4\\\" colnum=\\\"4\\\"\\u003e\\u003c/div\\u003e \\u003cdiv align=\\\"left\\\" class=\\\"colspec\\\" colname=\\\"c5\\\" colnum=\\\"5\\\"\\u003e\\u003c/div\\u003e \\u003cthead\\u003e \\u003ctr\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c1\\\"\\u003e\\u0026nbsp;\\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003eQuestion\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eNo\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eYes\\u003c/p\\u003e \\u003c/th\\u003e \\u003cth align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eUncertain\\u003c/p\\u003e \\u003c/th\\u003e \\u003c/tr\\u003e \\u003c/thead\\u003e \\u003ctbody\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e1.1\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eSoftware as a medical device (SaMD)\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003eHas the FDA cleared or approved the AI as SaMD\\u003csup\\u003ea\\u003c/sup\\u003e?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThe\\u003c/em\\u003e b\\u003cem\\u003eusiness owner is responsible for producing the FDA letter.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eContinue to 1.2\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eContinue to 1.3\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e1.2\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eSaMD\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003eCould the software meet the FDA definition of software as a device?\\u003c/p\\u003e \\u003cp\\u003ea. The AI acquires, processes, or analyzes a medical image or signal related to a patient\\u0026rsquo;s health. \\u003cem\\u003eIf this statement is TRUE, answer \\\"YES\\\".\\u003c/em\\u003e\\u003c/p\\u003e \\u003cp\\u003eb. The AI displays medical information about a patient, study or guideline. \\u003cem\\u003eIf this statement is TRUE, answer \\\"NO\\\".\\u003c/em\\u003e\\u003c/p\\u003e \\u003cp\\u003ec. The AI provides recommendations to a health care professional about prevention, diagnosis, or treatment of a disease AND provides the basis for recommendations, so the health care professional is not relying solely on the AI output for decision making. \\u003cem\\u003eIf this statement is TRUE, answer \\\"NO\\\".\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eModerate risk \\u003c/p\\u003e \\u003cp\\u003eContinue to 2\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003cp\\u003eContinue to 2\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk \\u003c/p\\u003e \\u003cp\\u003eContinue to 2\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e1.3\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eSaMD\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs the FDA approval of the AI as SaMD \\u003cspan type=\\\"Underline\\\" class=\\\"Underline\\\" name=\\\"Emphasis\\\"\\u003efor the intended use\\u003c/span\\u003e within the healthcare organization?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThe vendor and/or business owner are responsible for providing the FDA confirmation letter and all supporting documentation or data to allow for this determination.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e2\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003ePotential for significant adverse effects\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003ePotential adverse effects are \\u003cspan type=\\\"Underline\\\" class=\\\"Underline\\\" name=\\\"Emphasis\\\"\\u003enotable\\u003c/span\\u003e and could have a \\u003cspan type=\\\"Underline\\\" class=\\\"Underline\\\" name=\\\"Emphasis\\\"\\u003esignificant negative impact\\u003c/span\\u003e on patients, teammates, individuals, or the enterprise?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThere should be adequate evidence of implementation in other similar settings to support a \\u0026lsquo;NO\\u0026rsquo; answer.\\u003c/em\\u003e\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThe business owner is responsible for identifying supporting documentation\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e3\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAdverse effects and workflows\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003ePotential adverse effects are not minor but \\u003cspan type=\\\"Underline\\\" class=\\\"Underline\\\" name=\\\"Emphasis\\\"\\u003eare adequately addressed\\u003c/span\\u003e by workflows to mitigate/control the risk?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e4\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eNet benefit\\u003c/b\\u003e: \\u003c/p\\u003e \\u003cp\\u003eThere is substantial evidence that supports the benefits outweigh the risks that are expected from AI implementation?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eEvidence should include implementation in other similar settings to support a \\u0026lsquo;YES\\u0026rsquo; answer.\\u003c/em\\u003e\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eThe business owner is responsible for identifying supporting documentation.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e5\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAI features, equity in depth\\u003c/b\\u003e: \\u003c/p\\u003e \\u003cp\\u003eIf the AI uses features that include characteristics\\u003csup\\u003eb\\u003c/sup\\u003e historically used to discriminate, then adequate evidence is provided for how they influence the output in the context of the intended use?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e6\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAI output, equity in depth\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003eAdequate evidence is provided that the AI solution performs well in all key subgroups?\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; \\u003cem\\u003eE.g., a model appropriately ranks patients according to risk and does not systematically underestimate or overestimate risk.\\u003c/em\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk \\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e7\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eAccess, equity\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003eIs the AI system equally accessible to those who may benefit?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e8\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eMedical billing, coding, human resource\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003ea. Is an output of the AI that is related to medical billing or medical coding made part of a patient\\u0026rsquo;s permanent record or released to a third party without the intervention of a human?\\u003c/p\\u003e \\u003cp\\u003eb. Does the AI rank or categorize applicants or teammates for an intended use that is HR related?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e9\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003ePrivacy/transparency\\u003c/b\\u003e:\\u003c/p\\u003e \\u003cp\\u003ea. Does the AI solution record an individual without their knowledge?\\u003c/p\\u003e \\u003cp\\u003eb. Is the organization ethically obligated to provide an explicit explanation that AI is being used or need to consent that AI is being used, but that is not part of solution or workflow? (e.g., based on potential risk(s) or if no human is in the loop)\\u003c/p\\u003e \\u003cp\\u003ec. Does the AI solution analyze personal data that may lead to profiling or categorizing of individuals (excluding risk scoring for clinical diagnosis or clinical workflow prioritization)?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e10\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eDevelopment and validity\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eTransparent reporting of development and validation steps is available AND no concerns are identified when evaluated against contemporary published AI reporting standards. \\u0026nbsp; \\u003cem\\u003eIf this statement is TRUE, answer \\\"YES\\\".\\u003c/em\\u003e\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Answer \\u0026lsquo;NO\\u0026rsquo;, if supporting evidence is insufficient.\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Answer \\u0026lsquo;NO\\u0026rsquo;, if concerns are present regarding the general validity of the model.\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Answer \\u0026lsquo;NO\\u0026rsquo; if the AI solution's methods or outputs are \\\"blackbox\\\"\\u003csup\\u003ec\\u003c/sup\\u003e and the AI implementation creates the potential for anything more than minor adverse effects on patients, employees, or individuals.\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e11\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eExternal performance\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003ea. Substantive evidence of external performance exists to the level that a local validation\\u0026nbsp;is not required? OR\\u003c/p\\u003e \\u003cp\\u003eb. The development and validation data/environment are expected to be so similar to the local data/environment that local confirmation is NOT necessary (e.g., radiology imaging)?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e12\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eHuman oversight\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs the AI solution directly or indirectly tied to workflow(s) that automate an action, documentation, or patient communication without human review, which may\\u0026nbsp;adversely affect patient health outcomes?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003eModerate risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colspan=\\\"5\\\" nameend=\\\"c5\\\" namest=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eCarry forward low-risk screening questions that are high risk\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003e13\\u003c/b\\u003e\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eSensitive data\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the AI interface with data that may require special consideration?\\u003c/p\\u003e \\u003cp\\u003ea. Recording individuals\\u003c/p\\u003e \\u003cp\\u003eb. Facial recognition\\u003c/p\\u003e \\u003cp\\u003ec. Fingerprints\\u003c/p\\u003e \\u003cp\\u003ed. Genetic data\\u003c/p\\u003e \\u003cp\\u003ee. Claims/payor data\\u003c/p\\u003e \\u003cp\\u003ef. Other sensitive\\u0026nbsp;data\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e14\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eTrust\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eIs there reasonable potential that the AI may negatively impact trust between provider (or health system) and patient(s)?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e15\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eVulnerability considerations\\u003c/b\\u003e\\u003csup\\u003e\\u003cb\\u003ed\\u003c/b\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the AI implementation intersect with any of the following healthcare settings/functions/populations:\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Beginning of life (pre, peri, neo-natal)\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; End of life (hospice, DNR/code status, palliative care, advance directives)\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Consent for treatment/research\\u003c/p\\u003e \\u003cp\\u003e\\u0026bull; Capacity for decision making\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c1\\\"\\u003e \\u003cp\\u003e16\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c2\\\"\\u003e \\u003cp\\u003e\\u003cb\\u003eOther concern(s)\\u003c/b\\u003e\\u003c/p\\u003e \\u003cp\\u003eDoes the reviewer have any other significant concerns about the AI not captured within the low-risk screen?\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c3\\\"\\u003e \\u003cp\\u003en/a\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c4\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003ctd align=\\\"left\\\" colname=\\\"c5\\\"\\u003e \\u003cp\\u003eHigh risk\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003ctr\\u003e \\u003ctd align=\\\"left\\\" colspan=\\\"5\\\" nameend=\\\"c5\\\" namest=\\\"c1\\\"\\u003e \\u003cp\\u003eAbbreviations: FDA\\u0026thinsp;=\\u0026thinsp;Food and Drug Administration; AI\\u0026thinsp;=\\u0026thinsp;Artificial Intelligence; n/a\\u0026thinsp;=\\u0026thinsp;not available\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003ea\\u003c/sup\\u003eSaMD: Software as Medical Device (\\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttps://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software\\u003c/span\\u003e\\u003cspan address=\\\"https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e)\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003eb\\u003c/sup\\u003ePROGRESS-Plus: place of residence,\\u0026nbsp;race/ethnicity/culture/language, occupation, gender/sex,\\u0026nbsp;religion,\\u0026nbsp;education, socioeconomic status, social\\u0026nbsp;capital, personal characteristics associated with discrimination.\\u003csup\\u003e\\u003cspan citationid=\\\"CR28\\\" class=\\\"CitationRef\\\"\\u003e28\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003ec\\u003c/sup\\u003e\\\"Blackbox\\\" could mean either the information is proprietary and not shared or a deep learning model, which due to its complexity cannot be understood.\\u003c/p\\u003e \\u003cp\\u003e\\u003csup\\u003ed\\u003c/sup\\u003eVulnerability: The\\u0026nbsp;conditions determined by physical, social, economic, and environmental factors or processes which increase the susceptibility of an individual, a community, assets, or systems to the impacts of hazards\\u0026rdquo; (World Health Organization)\\u003c/p\\u003e \\u003c/td\\u003e \\u003c/tr\\u003e \\u003c/tbody\\u003e \\u003c/colgroup\\u003e \\u003c/table\\u003e\\u003c/div\\u003e \\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec15\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eRisk categorization and recommendations\\u003c/h2\\u003e \\u003cp\\u003eAfter application of the low-risk screening questions, the in-depth review questions (if necessary), and completion of the AI Governance committee review (if necessary), the proposed solution is assigned a final risk category, and a FAIR-AI Summary Statement is completed (an example is presented in Supplementary Box 1). At this point, an AI solution may need to go through other traditional governance requirements like a cyber security review, financial approvals, etc. If the AI solution ultimately is designated to move forward with implementation, then the data science team and business owners collaboratively develop a Safe AI Plan as outlined below.\\u003c/p\\u003e \\u003c/div\\u003e \\u003cdiv id=\\\"Sec16\\\" class=\\\"Section2\\\"\\u003e \\u003ch2\\u003eMonitoring and transparency requirements \\u0026ndash; Safe AI Plan\\u003c/h2\\u003e \\u003cp\\u003eImplemented AI solutions need continuous monitoring as they may fail to adapt to new data or practice changes, which can lead to inaccurate results and increasing bias over time.\\u003csup\\u003e\\u003cspan citationid=\\\"CR37\\\" class=\\\"CitationRef\\\"\\u003e37\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR38\\\" class=\\\"CitationRef\\\"\\u003e38\\u003c/span\\u003e\\u003c/sup\\u003e Similarly, when AI solutions are made readily available in workflows, it becomes easier for the solution to be used outside of its approved intended use case, which may change its inherent risk profile. For these reasons, FAIR-AI requires a monitoring plan for every deployed AI solution consisting of an attestation by the business owner at regular intervals. The attestation affirms that: (i) the deployment is still aligned with the approved use case; (ii) the underlying data and related workflows have not substantially changed; (iii) the AI solution is delivering the expected benefit(s); (iv) no unforeseen risks have been identified; and (v) there are no concerns noted related to new regulations. If the original FAIR-AI review identified specific risks, then the attestation also includes an approach to evaluate each risk along with metrics (if applicable). These evaluation metrics may range from repeating a standard model performance evaluation to obtaining periodic end-user feedback on accuracy (e.g., for a generative AI solution).\\u003c/p\\u003e \\u003cp\\u003eIn addition to monitoring, all solutions categorized as high risk also require an AI Label (Fig.\\u0026nbsp;\\u003cspan refid=\\\"Fig3\\\" class=\\\"InternalRef\\\"\\u003e3\\u003c/span\\u003e) and end-user education at regular intervals. Finally, in situations where an end-user could potentially not be aware they are interacting with AI instead of a human, the business owner must design implementation workflows that create transparency for the end-user (e.g., an alert, disclaimer, or consent as applicable).\\u003c/p\\u003e \\u003cp\\u003e \\u003c/p\\u003e \\u003c/div\\u003e\"},{\"header\":\"Discussion\",\"content\":\"\\u003cp\\u003eHealth systems are under growing pressure to adopt an increasingly wide array of AI solutions some of which have enormous potential to transform healthcare, but many also introduce complex potential risks. The FAIR-AI framework described in this paper offers a prescriptive, practical, and scalable approach for evaluating AI solutions for use in healthcare. We have distilled the approach into a concise set of questions that a data science team member can use to quickly triage AI solutions, triggering a more time-intensive, rigorous review only when necessary. This practical approach is necessary given the volume of new solutions released and as AI becomes more ubiquitous across healthcare. By establishing formal review criteria and a consistent risk assessment process, institutions can ensure well-documented, defensible recommendations. Ultimately, by implementing FAIR-AI or a similar framework, health systems can foster a culture that upholds high standards for both internally and vendor-developed AI solutions, protecting patients and the care team, while being an early adopter harnessing actual AI benefits.\\u003c/p\\u003e \\u003cp\\u003eThere are many challenges to implementing and maintaining the framework we have developed. Successful implementation requires support from institutional leadership, along with the allocation of resources to maintain documentation, manage new requests, and ensure proper monitoring. Team members tasked with screening requests must be empowered to reject requests for solutions that do not provide adequate documentation for a thorough review, otherwise, the process may become slow and inefficient as they search for information. In our early experience, we have found many AI solutions lack the evidence needed to support implementation and first require further research or pilot testing, which demands substantial resources from either the health system or the vendor. Generative AI solutions present significant challenges when they intersect with patient care, particularly around the difficulty in explaining how a tool functions, the opaque nature of the data used for training, the lack of standardized performance, the extensive manual effort required to review output, the need for infrastructure to obtain user-feedback, and mechanisms for reporting inaccuracies. An often overlooked but critical challenge to the responsible implementation of AI is the significant training required for both evaluators and end-users. Several recently published guidelines provide structured approaches for assessing the reliability and transparency of large language models in healthcare. We recognize the importance of these emerging frameworks and plan to expand our AI evaluation framework to incorporate relevant elements from them. However, integrating these considerations will take time, as adapting existing validation strategies for generative AI requires careful refinement to ensure a practical, efficient, and reproducible process that aligns with stakeholder needs.\\u003csup\\u003e\\u003cspan citationid=\\\"CR39\\\" class=\\\"CitationRef\\\"\\u003e39\\u003c/span\\u003e,\\u003cspan citationid=\\\"CR40\\\" class=\\\"CitationRef\\\"\\u003e40\\u003c/span\\u003e\\u003c/sup\\u003e\\u003c/p\\u003e \\u003cp\\u003eAt our organization, we plan to review and adapt FAIR-AI at least annually, due to the rapid changes in the field and regulatory environment. For example, AI tools themselves are being used increasingly to monitor other AI solutions for safety, and future iterations of FAIR-AI will need to account for this evolving area. As AI solutions become pervasive across most workflows, all teammates play a role in being vigilant with an awareness of AI’s inherent limitations, security risks, and ethical considerations. To address this need to democratize responsibility, we are developing accompanying education that will enhance our organization’s responsible AI culture.\\u003c/p\\u003e \\u003cp\\u003eThere are numerous limitations to our approach to evaluating AI solutions as described in this paper. Our evaluation and monitoring processes require a significant commitment of time and resources. Some health systems may choose to rely only on evaluations provided by other entities, which reduces the burden on the health system and speeds up the adoption of new AI tools; however, this may introduce inherent bias and conflicts of interest. While the screening and in-depth review questions provide a structured approach, they are not exhaustive, and the effectiveness of the framework depends on the diligence and expertise of the evaluators. Additionally, this framework will require that organizations make modifications to meet their needs and risk tolerance and to ensure alignment with local regulatory requirements.\\u003c/p\\u003e \\u003cp\\u003eFAIR-AI provides a practical template for health systems to adopt a process for the rigorous evaluation and monitoring of AI solutions. The prescriptive framework guided by explicit criteria is intentionally designed for health systems to use at the speed and scale required in real-world settings. This framework will enable institutions to carefully balance the desire to adopt innovative solutions while maintaining the highest standards for patient and care team safety.\\u003c/p\\u003e \"},{\"header\":\"Methods\",\"content\":\"\\u003ch2\\u003eBest practices - scoping review\\u003c/h2\\u003e\\u003cp\\u003eFor the scoping review, we employed a pragmatic approach, utilizing Google Scholar as the primary search engine to locate pertinent published frameworks and papers. Search terms included: \\u003cem\\u003eframework, guideline, evaluation, monitoring, transparency, explainability, artificial intelligence, validation, informatics, clinical decision support, ethics, equity, regulatory, legal, usefulness, risks, benefits, implementation, deployment, predictive model, machine learning, clinical utility, health.\\u003c/em\\u003e Additionally, we incorporated institutional guidelines from the National Institute of Standards and Technology (NIST) and the U.S. Food and Drug Administration (FDA) and conducted citation tracking to identify influential works.\\u003c/p\\u003e\\u003ch2\\u003eStakeholder needs - interviews\\u003c/h2\\u003e\\u003cp\\u003eFrom March to April 2024, we conducted semi-structured interviews with executive leadership (N = 3), senior risk, compliance, and legal leaders (N = 6), data developers (N = 4), providers (N = 5), and patients (N = 5) from across our health system. We utilized purposive sampling methods to ensure we obtained stakeholder feedback from the five user domains (e.g., executive leadership) that we felt would be most impacted by the implementation of an AI framework.\\u003csup\\u003e\\u003cspan citationid=\\\"CR39\\\" class=\\\"CitationRef\\\"\\u003e39\\u003c/span\\u003e\\u003c/sup\\u003e Interviewees were identified by members of the study team. An interview guide was collaboratively developed by the study team, which included physicians, faculty, and health system leaders with expertise in ethics, equity, data science, and care delivery. Each interview lasted approximately 30 minutes, was completed via telephone or videoconference, and was facilitated by a qualitatively trained faculty member (JK). Interviews were audio recorded and transcribed verbatim. Transcripts were analyzed using both inductive and deductive coding methodologies, with thematic analysis employed to identify and organize emergent themes in the data. Three members of the study team collaboratively developed the coding dictionary (BJW, JK, AM), with the qualitative lead (JK) independently coding all transcripts and bringing any questions back to the team for review.\\u003c/p\\u003e\\u003ch2\\u003eExpert consensus - design workshop\\u003c/h2\\u003e\\u003cp\\u003e We convened a half-day, in-person workshop to synthesize the best practices identified from the literature review, the priorities outlined by stakeholders, and the consensus recommendations from a diverse team of subject matter experts. Workshop participants included individuals with expertise in legal affairs, regulatory compliance, cyber security, ethics, clinical care, clinical informatics, data science, and research (N = 33). As the starting point for the workshop, the primary project team created a draft framework outline. This outline, along with background information, pertinent literature, and summaries of stakeholder needs, were shared with attendees for review prior to the meeting.\\u003c/p\\u003e\\u003ch2\\u003eEthical considerations\\u003c/h2\\u003e\\u003cp\\u003e The study was approved by the Wake Forest University Health Sciences Institutional Review Board (IRB00109544).\\u003c/p\\u003e\"},{\"header\":\"Declarations\",\"content\":\"\\u003cp\\u003e\\u003cstrong\\u003eAcknowledgments\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eThe study was supported by the Duke Endowment under award number AWD00002292. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Duke Endowment. We would like to express our gratitude to Dr. Reid Blackman for his valuable feedback on the design of the framework and Sally Baek and Michael Johnson from Atrium Health for their critical support with organizing the design workshop.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eData availability\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eThe datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request. Interview data are not made publicly available to protect the confidentiality of the interviewees, including senior leader participants.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eAuthor contributions\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eAM and BJW supervised the study. Members of the FAIR-AI Consortium contributed to the conception and design of the study. AB, SC, PC, AC, MG, TH, MI, JK, AK, AM, HMN, MP, YJT, and BJW performed acquisition, analysis, and interpretation of the data and drafted and revised the manuscript.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eCompeting interests\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eThe authors declare no competing interests.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003cstrong\\u003eFAIR-AI Consortium\\u003c/strong\\u003e\\u003c/p\\u003e\\n\\u003cp\\u003eOguz Akbilgic, PhD\\u003csup\\u003e9\\u003c/sup\\u003e, Katie Barr, MSN, RN\\u003csup\\u003e10\\u003c/sup\\u003e, Amy Bovi, MA\\u003csup\\u003e5\\u003c/sup\\u003e, Alicia Bowers, JD\\u003csup\\u003e11\\u003c/sup\\u003e, Rikki Caffrey, MA, MS\\u003csup\\u003e12\\u003c/sup\\u003e, Michael S. Carroll, PhD\\u003csup\\u003e4\\u003c/sup\\u003e, Shih-Hsiung Chou, PhD\\u003csup\\u003e4\\u003c/sup\\u003e, Matthew CiRullo, DO, MBA\\u003csup\\u003e13\\u003c/sup\\u003e, Patricia Corn, MSN, RN\\u003csup\\u003e7\\u003c/sup\\u003e, Audrey Cuison, MS\\u003csup\\u003e8\\u003c/sup\\u003e, Stephen M. Downs, MD, MS,\\u003csup\\u003e14\\u003c/sup\\u003e Mary Gagen, MBA\\u003csup\\u003e8\\u003c/sup\\u003e, Natalie Hardy, BA\\u003csup\\u003e5\\u003c/sup\\u003e, Timothy Hetherington, MS\\u003csup\\u003e4\\u003c/sup\\u003e, Jason Heuay, MS\\u003csup\\u003e4\\u003c/sup\\u003e, McKenzie Isreal, MPH\\u003csup\\u003e2\\u003c/sup\\u003e, Kristina Katzovitz, MD\\u003csup\\u003e15\\u003c/sup\\u003e, Eric Kirkendall, MD, MBI\\u003csup\\u003e16\\u003c/sup\\u003e, Justin Kramer, PhD, MAT\\u003csup\\u003e6\\u003c/sup\\u003e, Andrew Kuzma, PhD\\u003csup\\u003e5\\u003c/sup\\u003e, Elsie Lindgren, MBA, BSN, RN\\u003csup\\u003e17\\u003c/sup\\u003e, Lindsey Lonergan, JD\\u003csup\\u003e18\\u003c/sup\\u003e, Elissa McKinley, BS\\u003csup\\u003e19\\u003c/sup\\u003e, Andrew McWilliams, MD, MPH\\u003csup\\u003e3\\u003c/sup\\u003e, Hieu M. Nguyen, MS\\u003csup\\u003e2\\u003c/sup\\u003e, Nicholas M. Pajewski, PhD\\u003csup\\u003e1\\u003c/sup\\u003e, \\u0026nbsp;Matt Pallini, MS\\u003csup\\u003e4\\u003c/sup\\u003e, Laura Sak-Castellano, BS\\u003csup\\u003e20\\u003c/sup\\u003e, Erika Setliff, DNP, RN\\u003csup\\u003e21\\u003c/sup\\u003e, Yhenneko J. Taylor, PhD\\u003csup\\u003e2\\u003c/sup\\u003e, Brian J. Wells, MD, PhD\\u003csup\\u003e1\\u003c/sup\\u003e, Gabe Wright, JD\\u003csup\\u003e18\\u003c/sup\\u003e.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e9\\u003c/sup\\u003eDepartment of Cardiovascular Medicine, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e10\\u003c/sup\\u003eChief Nursing Informatics Officer, Advocate Health, Milwaukee, Wisconsin, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e11\\u003c/sup\\u003eInnovation and Commercialization, Advocate Health, Charlotte, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e12\\u003c/sup\\u003eClinical Ethics, Advocate Health, Chicago, Illinois, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e13\\u003c/sup\\u003eFamily Medicine, Atrium Health, Charlotte, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e14\\u003c/sup\\u003eDepartment of Pediatrics, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e15\\u003c/sup\\u003eChief Medical Information Officer, Advocate Health, Oakbrook, Illinois, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e16\\u003c/sup\\u003eChief Medical Information Officer \\u0026amp; Chief Information Officer, Advocate Health, Winston-Salem, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e17\\u003c/sup\\u003ePatient Safety, Advocate Health, Milwaukee, Wisconsin, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e18\\u003c/sup\\u003eOffice of the General Counsel, Atrium Health, Charlotte, North Carolina, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e19\\u003c/sup\\u003eCybersecurity Governance, Risk and Compliance, Advocate Health, Milwaukee, Wisconsin, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e20\\u003c/sup\\u003eAudit Services and Enterprise Risk Management, Advocate Health, Oak Brook, Illinois, USA.\\u003c/p\\u003e\\n\\u003cp\\u003e\\u003csup\\u003e21\\u003c/sup\\u003eVirtual Critical Care (Southeast Region Critical Care), Atrium Health, Charlotte, North Carolina, USA.\\u003c/p\\u003e\"},{\"header\":\"References\",\"content\":\"\\u003col\\u003e\\u003cli\\u003e\\u003cspan\\u003eRajpurkar, P., Chen, E., Banerjee, O. \\u0026amp; Topol, E. J. AI in health and medicine. Nat. Med. 28, 31\\u0026ndash;38 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eDedehayir, O. \\u0026amp; Steinert, M. The hype cycle model: A review and future directions. Technol. Forecast. Soc. Change 108, 28\\u0026ndash;41 (2016).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eShortliffe, E. H. \\u0026amp; Buchanan, B. G. A model of inexact reasoning in medicine. Math. Biosci. 23, 351\\u0026ndash;379 (1975).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eThirunavukarasu, A. J. \\u003cem\\u003eet al.\\u003c/em\\u003e Large language models in medicine. Nat. Med. 29, 1930\\u0026ndash;1940 (2023).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eBirkstedt, T., Minkkinen, M., Tandon, A. \\u0026amp; M\\u0026auml;ntym\\u0026auml;ki, M. AI governance: themes, knowledge gaps and future agendas. Internet Res. 33, 133\\u0026ndash;167 (2023).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eTaeihagh, A. Governance of artificial intelligence. Policy Soc. 40, 137\\u0026ndash;157 (2021).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eM\\u0026auml;ntym\\u0026auml;ki, M., Minkkinen, M., Birkstedt, T. \\u0026amp; Viljanen, M. Defining organizational AI governance. AI Ethics 2, 603\\u0026ndash;609 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eBedoya, A. D. \\u003cem\\u003eet al.\\u003c/em\\u003e A framework for the oversight and local deployment of safe and high-quality prediction models. J. Am. Med. Inform. Assoc. JAMIA 29, 1631\\u0026ndash;1636 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eReddy, S. \\u003cem\\u003eet al.\\u003c/em\\u003e Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. 28, e100444 (2021).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eVollmer, S. \\u003cem\\u003eet al.\\u003c/em\\u003e Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 368, l6927 (2020).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003evan der Vegt, A. H. \\u003cem\\u003eet al.\\u003c/em\\u003e Implementation frameworks for end-to-end clinical AI: derivation of the SALIENT framework. J. Am. Med. Inform. Assoc. JAMIA 30, 1503\\u0026ndash;1515 (2023).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eCollins, G. S. \\u003cem\\u003eet al.\\u003c/em\\u003e TRIPOD\\u0026thinsp;+\\u0026thinsp;AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 385, e078378 (2024).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eCollins, G. S., Reitsma, J. B., Altman, D. G. \\u0026amp; Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 13, 1\\u0026ndash;10 (2015).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eSteyerberg, E. W. \\u003cem\\u003eet al.\\u003c/em\\u003e Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiol. Camb. Mass 21, 128 (2010).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eBotchkarev, A. A New Typology Design of Performance Metrics to Measure Errors in Machine Learning Regression Algorithms. Interdiscip. J. Inf. Knowl. Manag. 14, 045\\u0026ndash;076 (2019).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eMoons, K. G. M., Altman, D. G., Vergouwe, Y. \\u0026amp; Royston, P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606 (2009).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eMoons, K. G. M. \\u003cem\\u003eet al.\\u003c/em\\u003e Risk prediction models: II. External validation, model updating, and impact assessment. Heart 98, 691\\u0026ndash;698 (2012).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003ePark, Y.-J. \\u003cem\\u003eet al.\\u003c/em\\u003e Assessing the research landscape and clinical utility of large language models: a scoping review. BMC Med. Inform. Decis. Mak. 24, 72 (2024).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eBandi, A., Adapa, P. V. S. R. \\u0026amp; Kuchi, Y. E. V. P. K. The Power of Generative AI: A Review of Requirements, Models, Input\\u0026ndash;Output Formats, Evaluation Metrics, and Challenges. Future Internet 15, 260 (2023).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eWiens, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Do no harm: a roadmap for responsible machine learning for health care. Nat. Med. 25, 1337\\u0026ndash;1340 (2019).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eScott, I., Carter, S. \\u0026amp; Coiera, E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health Care Inform. 28, e100251 (2021).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eKappen, T. H. \\u003cem\\u003eet al.\\u003c/em\\u003e Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn. Progn. Res. 2, 11 (2018).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eOsheroff, J. A. \\u003cem\\u003eet al.\\u003c/em\\u003e A Roadmap for National Action on Clinical Decision Support. J. Am. Med. Inform. Assoc. JAMIA 14, 141 (2007).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eBlackman, R. \\u003cem\\u003eEthical Machines: Your Concise Guide to Totally Unbiased, Transparent, and Respectful AI\\u003c/em\\u003e. (Harvard Business Review Press, 2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eDankwa-Mullan, I. \\u003cem\\u003eet al.\\u003c/em\\u003e A Proposed Framework on Integrating Health Equity and Racial Justice into the Artificial Intelligence Development Lifecycle. J. Health Care Poor Underserved 32, 300\\u0026ndash;317 (2021).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003ePaulus, J. K. \\u0026amp; Kent, D. M. Race and Ethnicity: A Part of the Equation for Personalized Clinical Decision Making? Circ. Cardiovasc. Qual. Outcomes 10, e003823 (2017).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003ePaulus, J. K. \\u0026amp; Kent, D. M. Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities. NPJ Digit. Med. 3, 99 (2020).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eO\\u0026rsquo;Neill, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J. Clin. Epidemiol. 67, 56\\u0026ndash;64 (2014).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eLiu, X. \\u003cem\\u003eet al.\\u003c/em\\u003e The medical algorithmic audit. Lancet Digit. Health 4, e384\\u0026ndash;e397 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eVasey, B. \\u003cem\\u003eet al.\\u003c/em\\u003e Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 377, e070904 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eZerilli, J., Bhatt, U. \\u0026amp; Weller, A. How transparency modulates trust in artificial intelligence. Patterns 3, (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003ede Hond, A. A. H. \\u003cem\\u003eet al.\\u003c/em\\u003e Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. NPJ Digit. Med. 5, 2 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eCrossnohere, N. L., Elsaid, M., Paskett, J., Bose-Brill, S. \\u0026amp; Bridges, J. F. P. Guidelines for Artificial Intelligence in Medicine: Literature Review and Content Analysis of Frameworks. J. Med. Internet Res. 24, e36823 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eKramer, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Developing a Framework for the Review and Oversight of Artificial Intelligence at a Large Healthcare Enterprise: Assessing the Needs and Priorities of Senior Health System Leadership, Providers, and Community Stakeholders. in (AcademyHealth, 2024).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eMatheny, M. E., Whicher, D. \\u0026amp; Thadaney Israni, S. Artificial Intelligence in Health Care: A Report From the National Academy of Medicine. JAMA 323, 509\\u0026ndash;510 (2020).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eFischhoff, B., Watson, S. R. \\u0026amp; Hope, C. Defining risk. Policy Sci. 17, 123\\u0026ndash;139 (1984).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eFeng, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare. Npj Digit. Med. 5, 1\\u0026ndash;9 (2022).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eLu, J. \\u003cem\\u003eet al.\\u003c/em\\u003e Learning under Concept Drift: A Review. Preprint at \\u003cspan class=\\\"ExternalRef\\\"\\u003e\\u003cspan class=\\\"RefSource\\\"\\u003ehttp://arxiv.org/abs/2004.05785\\u003c/span\\u003e\\u003cspan address=\\\"http://arxiv.org/abs/2004.05785\\\" targettype=\\\"URL\\\" class=\\\"RefTarget\\\"\\u003e\\u003c/span\\u003e\\u003c/span\\u003e (2020).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eGallifant, J. et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat. Med. 31, 60\\u0026ndash;69 (2025).\\u003c/span\\u003e\\u003c/li\\u003e \\u003cli\\u003e\\u003cspan\\u003eTam, T. Y. C. et al. A framework for human evaluation of large language models in healthcare derived from literature review. Npj Digit. Med. 7, 1\\u0026ndash;20 (2024).\\u003c/span\\u003e\\u003c/li\\u003e\\u003c/ol\\u003e\"}],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":true,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":false,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":true,\"isAuthorSuppliedPdf\":false,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":false,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"npj-digital-medicine\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"npjdigitalmed\",\"sideBox\":\"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)\",\"snPcode\":\"41746\",\"submissionUrl\":\"https://submission.springernature.com/new-submission/41746/3\",\"title\":\"npj Digital Medicine\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"NPJ\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true},\"keywords\":\"Artificial Intelligence, Framework, Practice Guideline, Healthcare, Biomedical Ethics, Costs and Benefits, Safety, Harm, Risk, Regulations, Predictive Model, Software, Algorithms, Clinical Decision Support Systems, Clinical Informatics, Decision Analysis, Health Equity, Data Science\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-5975624/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-5975624/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"Health systems face the challenge of balancing innovation and safety to responsibly implement artificial intelligence (AI) solutions. The rapid proliferation, growing complexity, ethical considerations, and rising demand for these tools require timely and efficient processes for rigorous evaluation and ongoing monitoring. Current AI evaluation frameworks often lack the practical guidance for health systems to address these challenges. To fill this gap, we developed a prescriptive evaluation and governance framework informed by a literature review, in-depth interviews with key stakeholders, including patients, and a multidisciplinary design workshop. The resulting framework provides health systems an outline of the resources, structures, criteria, and template documents to enable pre-implementation evaluation and post-implementation monitoring of AI solutions. Health systems will need to treat this or any alternative framework as a living document to maintain relevance and effectiveness as the AI landscape and regulations continue to evolve.\",\"manuscriptTitle\":\"A Practical and Prescriptive Framework for Appropriate Implementation and Review of Artificial Intelligence (FAIR-AI) in Healthcare\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-03-18 11:58:30\",\"doi\":\"10.21203/rs.3.rs-5975624/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0},{\"type\":\"decision\",\"content\":\"Revision requested\",\"date\":\"2025-03-19T11:06:43+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-03-11T15:32:52+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-03-10T14:30:23+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"editorInvitedReview\",\"content\":\"\",\"date\":\"2025-03-05T17:40:52+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"97816341418296316630051856123607318560\",\"date\":\"2025-02-19T15:50:35+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"234185929958973243004260563222010759880\",\"date\":\"2025-02-18T10:05:41+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewerAgreed\",\"content\":\"20249431612474957127778841139333989723\",\"date\":\"2025-02-17T14:34:11+00:00\",\"index\":\"hide\",\"fulltext\":\"\"},{\"type\":\"reviewersInvited\",\"content\":\"\",\"date\":\"2025-02-17T13:46:45+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"editorAssigned\",\"content\":\"\",\"date\":\"2025-02-07T14:06:43+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"checksComplete\",\"content\":\"\",\"date\":\"2025-02-07T05:08:33+00:00\",\"index\":\"\",\"fulltext\":\"\"},{\"type\":\"submitted\",\"content\":\"npj Digital Medicine\",\"date\":\"2025-02-06T17:41:52+00:00\",\"index\":\"\",\"fulltext\":\"\"}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"npj-digital-medicine\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":false,\"externalIdentity\":\"npjdigitalmed\",\"sideBox\":\"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)\",\"snPcode\":\"41746\",\"submissionUrl\":\"https://submission.springernature.com/new-submission/41746/3\",\"title\":\"npj Digital Medicine\",\"twitterHandle\":\"\",\"acdcEnabled\":true,\"dfaEnabled\":true,\"editorialSystem\":\"stoa\",\"reportingPortfolio\":\"NPJ\",\"inReviewEnabled\":true,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"6b510fbf-412c-451c-9382-59e33f468b17\",\"owner\":[],\"postedDate\":\"March 18th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"published-in-journal\",\"subjectAreas\":[{\"id\":45535213,\"name\":\"Biological sciences/Computational biology and bioinformatics/Machine learning\"},{\"id\":45535214,\"name\":\"Biological sciences/Computational biology and bioinformatics/Predictive medicine\"},{\"id\":45535215,\"name\":\"Health sciences/Health care/Health services\"},{\"id\":45535216,\"name\":\"Health sciences/Health care/Medical ethics\"},{\"id\":45535217,\"name\":\"Health sciences/Health care\"},{\"id\":45535218,\"name\":\"Physical sciences/Mathematics and computing/Information technology\"},{\"id\":45535219,\"name\":\"Physical sciences/Mathematics and computing/Software\"}],\"tags\":[],\"updatedAt\":\"2025-08-18T16:06:17+00:00\",\"versionOfRecord\":{\"articleIdentity\":\"rs-5975624\",\"link\":\"https://doi.org/10.1038/s41746-025-01900-y\",\"journal\":{\"identity\":\"npj-digital-medicine\",\"isVorOnly\":false,\"title\":\"npj Digital Medicine\"},\"publishedOn\":\"2025-08-11 15:58:05\",\"publishedOnDateReadable\":\"August 11th, 2025\"},\"versionCreatedAt\":\"2025-03-18 11:58:30\",\"video\":\"\",\"vorDoi\":\"10.1038/s41746-025-01900-y\",\"vorDoiUrl\":\"https://doi.org/10.1038/s41746-025-01900-y\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-5975624\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-5975624\",\"identity\":\"rs-5975624\",\"version\":[\"v1\"]},\"buildId\":\"XKTyCvWXoU3ODBz1xrDgd\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}