Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity

Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity Jessica Tschida, Amy Drahota This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1930423/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 16 May, 2023 Read the published version in Implementation Science Communications → Version 1 posted 6 You are reading this latest preprint version Abstract Background: Although evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children, they are often inconsistently implemented or not implemented in community settings where many autistic children receive usual care services. One blended multi-phased implementation strategy developed to support the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings is the Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments (ACT SMART Toolkit). Guided by an adapted version of the EPIS, the ACT SMART Toolkit is comprised multiple phases, each with steps and activities to be completed by agency implementation teams. In the present study, we evaluated implementation strategy fidelity, given the potential for important insights into the use of the toolkit and limited information on the phenomenon of implementation strategy fidelity more broadly. Methods: We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agency implementation teams. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level. We also conducted repeated measures ANOVAs to determine whether implementation strategy fidelity significantly differed by toolkit phase. Results: Overall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were high, with some variability by toolkit phase and activity, as well as by ASD community agency. There was a significant main effect of toolkit phase for dose ( F (2,8) = 10.93, MSE = .190, p = .005, η 2 = .73), such that dose was significantly lower during the preparation phase as compared to the implementation phase of the toolkit. Conclusions: Our instrumental case study evaluation of fidelity to the ACT SMART Toolkit demonstrated potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context. fidelity autism spectrum disorder blended implementation strategy case study implementation fidelity Figures Figure 1 Contributions to the literature Assessing implementation strategy fidelity is critical to advance the field of implementation science but is rarely evaluated in extant literature. In an instrumental case study, we found high but variable fidelity to a blended, multi-phased implementation strategy (the ACT SMART Toolkit) for increasing EBP use in ASD community agencies. These findings provide evidence that community-based providers utilize the ACT SMART Toolkit with fidelity, furthering the rationale that the toolkit is a promising strategy to increase EBP use within ASD community agencies. The paper contributes an innovative model of assessing implementation strategy fidelity, suggesting important influences of strategy content and context. Background Autism Spectrum Disorder. An autism spectrum disorder (ASD) affects approximately 1 in 44 children in the United States and has been identified as a public health concern estimated to cost $461 billion dollars a year for services and treatment by 2030 ( 1 – 3 ). ASD is characterized by social and communication difficulties as well as restricted and repetitive behaviors and interests, and commonly co-occurs with anxiety disorders, obsessive compulsive disorder, attention deficit hyperactivity disorder, and/or oppositional defiant disorder ( 4 – 6 ). Additionally, children on the autism spectrum have higher rates of behaviors such as self-injury, aggression, tantrums, and property destruction compared to neurotypical peers ( 7 – 9 ). Both the core features and co-occurring disorders and behaviors of ASD have been found to predict unsatisfactory outcomes in quality-of-life factors, including peer relationships, educational attainment, employment, and independent living in adulthood ( 5 , 10 , 11 ). Associations between autistic characteristics and unsatisfactory quality-of-life outcomes are also maintained by systemic barriers to inclusion of autistic individuals, such as societal stigma and lack of appropriate accommodations, in education, employment, and housing opportunities ( 12 – 14 ). The prevalence rate for ASD continues to grow dramatically as practices for diagnosis improve ( 3 , 15 ). However, despite their potential to improve outcomes for autistic youth and reduce individual and societal costs ( 16 – 18 ), barriers to community level identification and intervention remain ( 3 , 19 ). Although evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children, they are often inconsistently implemented or mis-implemented in community-based settings where many autistic children receive services ( 20 – 24 ). As a result, there is a considerable number of children on the autism spectrum not receiving practices empirically demonstrated to improve outcomes as part of their usual care. Thus, there is a need to identify, develop, and evaluate strategies facilitating the implementation of EBPs for ASD within community settings. ACT SMART Implementation Toolkit. Drahota and colleagues ( 25 , 26 ) developed a blended implementation strategy to support the implementation of EBPs for ASD in community settings: Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments (ACT SMART Toolkit). The ACT SMART Toolkit was developed through a review of existing implementation strategy evidence and by incorporating insight from a community-academic partnership ( 27 ). The ACT SMART Toolkit involves facilitation meetings led by trained ACT SMART facilitators and a web-based interface to guide ASD community agency leaders, supervisors, and providers that comprise agency implementation teams through phases of implementing an EBP ( 25 , 26 , 28 ). Drahota and colleagues ( 20 , 25 ) designed the ACT SMART Toolkit to have steps and activities that align with an adapted implementation model, the adapted Exploration, Adoption, Preparation, Implementation, Sustainment (EPIS) model ( 20 , 29 ). Overall, the toolkit guides ASD agency implementation teams to explore their agency’s receptivity to implementing a new EBP, identify and decide upon an EBP that meets their needs, prospectively plan to implement the EBP, implement the EBP, and finally evaluate implementation and develop a plan for sustainment (See Fig. 1; 25). Importantly, the ACT SMART Toolkit has been pilot tested with six ASD community-based agencies. Preliminary work by Drahota and colleagues (in preparation) suggests that the toolkit is feasible, acceptable, and useful to agency implementation teams ( 30 ). In addition, Sridhar and Drahota ( 31 ) found that the toolkit facilitates clinically meaningful changes in agency provider- and supervisor-reported EBP use. Moreover, Sridhar and colleagues ( 32 ) identified salient facilitators (i.e., facilitation, facilitation meetings, and phase specific activities) and salient barriers (i.e., website issues, perceived lack of resources, and contextual factors within ASD community agencies such as time constraints and funding) to the utilization of the ACT SMART Toolkit in the pilot study. Therefore, the next incremental, yet crucial, step in evaluating initial use of the ACT SMART Toolkit is to assess implementation strategy fidelity. Implementation Strategy Fidelity. Fidelity is a construct that assesses the extent to which individuals (e.g., providers) deliver a strategy as planned ( 33 – 35 ). Researchers have proposed components that contribute to fidelity include: ( 1 ) adherence to the outlined procedures, ( 2 ) proportion of the strategy received (i.e., dose), ( 3 ) extent of individual responsivity to the strategy (i.e., participant responsiveness), ( 4 ) quality of implementation, and ( 5 ) differentiation from unspecified procedures ( 36 , 37 ). Researchers have also proposed that quality and differentiation primarily capture the characteristics of an EBP being implemented whereas adherence, dose, and participant responsiveness hold relevance for implementation strategy fidelity ( 35 , 38 ). Dusenbury ( 36 ) defines adherence as the extent to which activities are consistent with the way a strategy is proposed, dose as the amount of strategy content received by participants, and participant responsiveness as the extent to which participants are engaged by and involved in the strategy. In relation to the ACT SMART Toolkit, participants would refer to the agency implementation teams (i.e., a group of individuals within an agency responsible for facilitating EBP implementation; 39). Fidelity is also considered dynamic and may be influenced by factors such as provider characteristics, the setting, and/or complexity of the strategy ( 35 , 40 ). Assessing implementation strategy fidelity can help implementation strategy developers further understand which components of an implementation strategy may be core functions needed to produce desired outcomes and which components may be adapted to account for varying contextual characteristics ( 41 – 43 ). Of course, this is contingent upon an ability to determine whether implementation of the strategy remained consistent with its underlying theory ( 44 , 45 ). Notably, increasing understanding about how implementation strategies work has been identified as a research priority within the field of dissemination and implementation science ( 46 – 48 ). Despite its importance, fidelity to implementation strategies has rarely been assessed; instead, research has often focused only on fidelity to the EBPs being implemented ( 35 , 49 ). Indeed, Slaughter et al. ( 35 ) conducted a scoping review that indicated no articles reporting fidelity to implementation strategies included definitions or conceptual frameworks for assessing implementation strategy fidelity. To our knowledge, few studies have examined fidelity to an implementation strategy, and only one recent study has used a guiding theoretical framework ( 49 , 50 ). Present Study. Using an instrumental case study approach to assess fidelity to the ACT SMART toolkit during its pilot study may be able to provide important insights into the use of the toolkit as well as the phenomenon of implementation strategy fidelity more broadly ( 51 ). Examining implementation strategy fidelity can provide insight into the overall potential for ASD community-based agencies to use the toolkit as planned and ultimately report greater use of EBPs. This information may be particularly useful for ASD community-based agencies, given potential competing priorities and contextual barriers to completing the toolkit in its entirety ( 32 ). Further, we provide one of the first process models to assess fidelity to a blended, multi-phased implementation strategy. This model may then inform a broader understanding of implementation strategy fidelity and contribute to underlying theory. Specifically, we addressed two key questions: What was fidelity to the ACT SMART Toolkit at an aggregate and individual agency level according to adherence, dose, and participant responsiveness during its pilot study? Was fidelity to the ACT SMART Toolkit significantly different by toolkit phases? Methods Participants. A total of six ASD community agencies located in Southern California were included in the pilot study of the ACT SMART toolkit. Four of the ASD community agencies were Applied Behavior Analysis (ABA) organizations, one was an ABA and mental health organization, and one agency was a Speech and Language Pathology organization. Five of the six ASD community agencies chose to adopt the EBP, Video Modeling, during the pilot study and complete all phases of the ACT SMART toolkit. One ABA agency chose not to adopt an EBP at the end of the adoption decision phase of the toolkit. Each ASD community agency developed implementation teams composed of agency staff (see Table 1 for implementation team demographic and discipline information). At least one agency leader was required for each implementation team. Eligibility criteria for agency leaders were: ( 1 ) holding the role of CEO, director, or leading decision-maker regarding treatment use at an ASD community agency eligible to participate in the ACT SMART pilot study, ( 2 ) willingness to participate in the pilot study for 1 year, and ( 3 ) agreement to provide feedback after completing each phase of the pilot study. The agency leader for each participating agency then invited up to four other agency staff members (i.e., supervisors and direct providers) to complete their agency’s implementation team. Eligibility criteria for implementation team members was to agree to complete the toolkit and provide feedback about its feasibility, acceptability, and utility. Table 1 Demographic and discipline information across implementation teams Agency Leaders ( n = 7) Supervisors ( n = 8) Direct Providers ( n = 1) Sex Assigned at Birth (Females) 100% 100% 100% Race White 100% 25% 100% Mixed Race - 25% - Prefer Not to Answer - 12.5% - Missing - 37% - Education Level Master’s Degree 42.9% 50% 100% Doctorate 57.1% 12.5% - Missing - 37% - Discipline Psychology 28.6% 25% - Behavior Specialist 28.6% 25% 100% Speech/Language/Communication 28.6% 12.5% - Education 14.3% - - Missing - 37% - Materials & Procedure. As part of the pilot study, a research assistant served as an independent observer and evaluated implementation teams’ fidelity using the Implementation Milestones form, adapted with permission from the Stages of Implementation Completion ( 52 ), and the ACT SMART Activity Fidelity form ( 53 ). The ACT SMART Implementation Milestones form required the independent observer to record a Yes or No answer (scored as 1 and 0, respectively) for whether activities during pre-implementation and phase 1 through phase 4 of the ACT SMART Toolkit were completed. Scores were converted into percentages to assist with interpretation. The ACT SMART Activity Fidelity form presented more detailed questions regarding completion of activities during Phase 2: Adoption; Phase 3: Preparation; and Phase 4: Implementation. The independent observer recorded a Yes or No answer (scored as 1 and 0, respectively) for whether implementation teams completed each activity and then rated the amount of the form completed using a 4-point Likert scale where 0 = “Nothing Completed”, 1 = “Minimally Completed (1–2 items)”, 2 = “Moderately Completed (3–4 items)”, and 3 = “Mostly/All Completed (5–6 items)”. In addition to the observational data collected using the ACT SMART Implementation Milestones form and the ACT SMART Activity Fidelity form, ACT SMART facilitators rated implementation team engagement using the ACT SMART Implementation Team Engagement Rating Scale that was created by the toolkit developers. Immediately after each facilitation meeting, the ACT SMART facilitator(s) rated implementation team engagement in ACT SMART activities and facilitation meetings since the last facilitation meeting occurred. Engagement ratings were completed using a 5-point Likert scale where 1 = “Not at all engaged”, 2 = “Slightly Engaged”, 3 = “Moderately Engaged”, 4 = “Very Engaged”, and 5 = “Extremely Engaged”. In the present study, we used the operational definitions from Dusenbury ( 36 ) and an overall scoring rubric for implementation strategy fidelity developed in Slaughter et al. ( 35 ) as the basis for using the ACT SMART Implementation Milestones form, ACT SMART Activity Fidelity form, and ACT SMART Implementation Team Engagement Rating Scale to assess implementation strategy fidelity via adherence, dose, and participant responsiveness, respectively. Analysis Plan. We used an instrumental case study approach to explore both fidelity to the ACT SMART Toolkit and potential generalizations to a broader underlying theory of implementation strategy fidelity. The Standards for Reporting Implementation Studies (StaRI) checklist was used to assist reporting given that the ACT SMART Toolkit is an implementation strategy developed to increase EBP use in ASD community agencies ( 54 ). First, we assessed adherence, dose, and participant responsiveness for the ACT SMART Toolkit overall as well as for each phase and activity of the toolkit. Utilizing the ACT SMART Implementation Milestones form, we assessed adherence via a Yes/No answer to whether implementation milestones were completed. Overall, by phase, and by activity, we calculated the average percentage of “Yes” answers for required toolkit activities. We assessed dose by analyzing Likert scales on the ACT SMART Activity Fidelity form evaluating how much of each activity was completed. Overall, by phase, and by activity, we calculated the mean dose rating. Finally, we assessed participant responsiveness by analyzing the Likert scales on the ACT SMART Implementation Team Engagement Rating Scale and used dates of completion to confirm phase. Overall and by phase, we calculated the mean participant responsiveness rating. We did not calculate the mean participant responsiveness rating by activity as ratings for engagement were only given by phase. We also calculated an average percent agreement on participant responsiveness ratings from facilitation meetings in which multiple facilitators were present. Lastly, we calculated overall, phase, and activity adherence, dose, and participant responsiveness for each agency implementation team. To evaluate whether adherence, dose, or participant responsiveness significantly differed by toolkit phase, we conducted repeated measures ANOVAs with toolkit phase as a within-subjects factor. We also conducted Bonferroni post-hoc tests and calculated effect sizes using local error terms. It should be noted that dose was not observed during phase 1 of the toolkit. Further, the one ASD community agency that chose not to adopt an EBP at the end of the adoption decision phase (Phase 2) of the toolkit did not have any implementation strategy fidelity variables observed during Phase 3 or Phase 4 of the toolkit. Results Aggregate Fidelity to the ACT SMART Toolkit. Agency implementation teams adhered to an overall average of 90% ( SD = 11.3%) of required ACT SMART Toolkit activities. Average adherence ranged from 74% ( SD = 19.5%) completion of required toolkit activities during the preparation phase of the toolkit to 100% ( SD = 0%) completion of required toolkit activities during the implementation phase of the toolkit (see Table 2 ). While completion rate for individual activities within phases was also relatively high across agencies, there was some variability. There were lower average completion rates for activities such as the benefit-cost estimator, gathering treatment materials, and developing adaptation and implementation plans compared to higher average completion rates for activities related to treatment evaluation, funding, and training. Table 2 Adherence to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team Aggregate M% ( SD ) Agency 1 M% ( SD ) Agency 2 M% ( SD ) Agency 3 a M% ( SD ) Agency 4 M% ( SD ) Agency 5 M% ( SD ) Agency 6 M% ( SD ) Overall Adherence Scores 90% (11.3) 90.8% (14.5) 93.3% (14.9) 91.7% (14.4) 89.5% (17.4) 88.0% (26.8) 85.3% (20.2) Pre-Implementation 100% 100% 100% 100% 100% 100% 100% Agency first contacted 100% 100 100 100 100 100 100 Agency interest indicated 100% 100 100 100 100 100 100 Agency recruitment meeting 100% 100 100 100 100 100 100 Orientation meeting attendance 100% 100 100 100 100 100 100 Phase 1: Exploration 83% (18.0) 66.7% (57.7) 66.7% (57.7) 100% 100% 100% 66.7% (57.7) Recruit for agency assessment 83% (40.8) 100 0 100 100 100 100 Agency assessment link sent 100% 100 100 100 100 100 100 Staff response rate ≥ 75% 67% (51.6) 0 100 100 100 100 0 Phase 2: Adoption 92% (17.8) 87.5% (35.4) 100% 75.0% (46.3) 87.5% (35.4) 100% 100% Treatment selection 100% 100 100 100 100 100 100 Evaluate fit 100% 100 100 100 100 100 100 Evaluate feasibility 100% 100 100 100 100 100 100 Evaluate clinical utility and validity 83% (40.8) 100 100 0 100 100 100 Evaluate training requirements 100% 100 100 100 100 100 100 Evaluate funding source 100% 100 100 100 100 100 100 Evaluate benefits and costs 50% (54.8) 0 100 0 0 100 100 Validate adoption decision 100% 100 100 100 a 100 100 100 Phase 3: Preparation 74% (19.5) 100% 100% 60% (54.8) 40% (54.8) 60% (54.8) Gather and review treatment materials 60% (54.8) 100 100 0 0 100 Evaluate prospective adaptations 80% (44.7) 100 100 100 0 100 Develop adaptation plan 50% (70.7) 100 N/A N/A 0 N/A Develop training plan 100% 100 100 100 100 100 Develop implementation plan 80% (44.7) 100 100 100 100 0 Phase 4: Implementation 100% 100% 100% 100% 100% 100% Carry out adaptation plan 100% 100 N/A N/A N/A N/A Carry out training plan 100% 100 100 100 100 100 Carry out implementation plan 100% 100 100 100 100 100 b Note. Adherence scoring range is 0–100%. a Agency implementation team made the decision to not adopt an EBP. b Agency implementation team created and carried out an implementation plan in Phase 4. Related to dose, the independent observer gave agency implementation teams an overall average rate falling between “Moderately Completed” to “Mostly/All Completed” ( M = 2.3, SD = .60). The lowest average dose rating was between “Minimally Completed” to “Moderately Completed” ( M = 1.7, SD = .60) during the preparation phase whereas the highest average dose rating was between “Moderately Completed” to “Mostly/All Completed” ( M = 2.9, SD = .01) during the implementation phase of the toolkit (see Table 3 ). Consistent with observations of adherence, there were lower average dose ratings for activities such as the benefit-cost estimator, gathering treatment materials, and developing adaptation and implementation plans compared to higher average completion rates for activities related to treatment evaluation, funding, training, and carrying out developed plans. Here, it should be noted that average dose ratings by activity could not be calculated for the implementation phase given that evaluation surveys during this phase were designed to be dynamic and capture completion of different sets of tasks by agency ( 30 ). Table 3 Dose to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team Aggregate M ( SD ) Agency 1 M ( SD ) Agency 2 M ( SD ) Agency 3 a M ( SD ) Agency 4 M ( SD ) Agency 5 M ( SD ) Agency 6 M ( SD ) Overall Dose Scores 2.3 (0.6) 2.7 (0.4) 2.7 (0.4) 1.5 (0) 2.3 (0.7) 2.3 (1.3) 2.3 (0.7) Phase 1: Exploration † † † † † † † Phase 2: Adoption 2.4 (0.6) 3.0 (0) 2.7 (0.8) 1.5 (1.6) 2.4 (1.1) 3.0 (0) 2.0 (1.3) Evaluate fit 3.0 (0) 3 3 3 3 3 3 Evaluate feasibility 3.0 (0) 3 3 3 3 3 3 Evaluate clinical utility and validity 2.5 (1.2) 3 3 0 3 3 3 Evaluate training requirements 2.3 (1.0) * 3 * 2 * 3 Evaluate funding source 2.3 (1.0) 3 1 3 3 3 1 Evaluate benefits and costs 1.4 (1.5) - 3 0 0 3 1 Validate adoption decision 2.0 (1.6) 3 3 0 a 3 3 0 Phase 3: Preparation 1.7 (0.6) 2.2 (1.1) 2.3 (1.0) 1.6 (1.5) 0.8 (1.3) 1.8 (1.5) Gather and review treatment materials 0.6 (0.6) 1 1 0 0 1 Evaluate prospective adaptations 2.4 (1.3) 3 3 3 0 3 Develop adaptation plan 1.0 (1.7) 3 N/A N/A 0 N/A Develop training plan 2.6 (0.9) 3 3 3 1 3 Develop implementation plan 1.6 (1.1) 1 2 2 3 0 Phase 4: Implementation 2.9 (0.1) 2.9 (0.4) 3.0 (0) 3.0 (0) 3.0 (0) 3.0 (0) Note. Dose scoring ranges from 0 (Nothing completed) to 3 (Mostly/All completed [5–6 items]). a Agency implementation team made the decision to not adopt an EBP, therefore, did not progress past Phase 2. † Denotes that this is not applicable for the fidelity domain. * Agency implementation team indicated that there were no training requirements while completing form. - Indicates missing data. For participant responsiveness, ACT SMART facilitators rated agency implementation teams an overall average corresponding to “Very Engaged” ( M = 4.0, SD = .50). The lowest average participant responsiveness rating was between “Slightly Engaged” and “Moderately Engaged” ( M = 2.3, SD = 1.0) during the adoption decision phase of the toolkit. The highest average participant responsiveness rating was between “Very Engaged” to “Extremely Engaged” ( M = 4.7, SD = .50) during the implementation phase (see Table 4 ). For facilitation meetings with multiple ACT SMART facilitators present, there was a 92.43% average agreement on participant responsiveness ratings. Table 4 Participant responsiveness to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team Aggregate M ( SD ) Agency 1 M ( SD ) Agency 2 M ( SD ) Agency 3 a M ( SD ) Agency 4 M ( SD ) Agency 5 M ( SD ) Agency 6 M ( SD ) Overall Participant Responsiveness Scores 4.0 (0.5) 4.1 (0.7) 4.6 (0.5) 3.3 (0.1) 3.8 (0.3) 4.5 (0.4) 3.8 (0.9) Phase 1: Exploration 3.8 (0.7) 4 4 3.25 3.5 5 3 Phase 2: Adoption 2.3 (1.0) 3.5 5 3.4 a 3.5 4.5 3.5 Phase 3: Preparation 3.5 (1.9) 3.8 4.5 4.0 4.1 3.6 Phase 4: Implementation 4.7 (0.5) 5 5 4 4.3 5 Note. Dose scoring ranges from 1 (Not at all engaged) to 5 (Extremely engaged). a Agency implementation team made the decision to not adopt an EBP, therefore, did not progress past Phase 2. Individual Agency Fidelity to the ACT SMART Toolkit. Across agencies, there was generally high adherence to toolkit activities; the lowest agency implementation team adhered to an overall average of 85.3% ( SD = 20.2%) of required toolkit activities (Table 2 ). While there was some variability in adherence across phases and activities by agency, there was no readily identifiable pattern of agencies consistently having lower or higher adherence compared to other agencies. Consistent with other results, the preparation phase appeared to have the lowest adherence ratings across agencies. Agencies also all had generally high dose ratings for toolkit activities, except for the one agency (Agency 3) that chose not to adopt an EBP at the end of the Phase 2: Adoption (Table 3 ). Like the ratings of adherence by agency, there was variability in dose ratings but no consistent identifiable patterns. Further, the preparation phase had the lowest dose ratings across agencies. Consistent with both observations of adherence and dose ratings across agencies, all agencies also had relatively high ratings of participant responsiveness (Table 4 ). The agency with the lowest average participant responsiveness rating was rated between “Moderately Engaged” to “Very Engaged” ( M = 3.3, SD = 0.1). However, in contrast to observations of adherence and dose ratings, agencies did not appear to have lower participant responsiveness during the preparation phase compared to other toolkit phases. Differences in Fidelity to the ACT SMART Toolkit by Toolkit Phase. Our repeated measures ANOVAs to compare implementation strategy fidelity variables (i.e., adherence, dose, participant responsiveness) across phases revealed a significant main effect of toolkit phase for dose ( F ( 2 , 8 ) = 10.93, MSE = .190, p = .005, η 2 = .73, 95% CI [.15, .84]). However, there was not a significant main effect of toolkit phase for either adherence ( F ( 3 , 12 ) = 2.43, MSE = .035, p = .116, η 2 = .38, 95% CI [0, .57]) or participant responsiveness ( F ( 3 , 12 ) = .2.31, MSE = .258, p = .128, η 2 = .37, 95% CI [0, .69]). Using the Bonferroni post-hoc tests with local error terms to further examine the significant main effect of toolkit phase on dose, we found that the average dose rating during the preparation phase (Phase 3) of the toolkit was significantly lower than the average dose rating during the implementation phase (Phase 4) of the toolkit ( d = 2.93, 95% CI [0.59, 5.20]. Discussion Fidelity to the ACT SMART Toolkit. Our investigation used an instrumental case study approach to evaluate implementation strategy fidelity to the ACT SMART Toolkit by assessing observational descriptive ratings of adherence, dose, and participant responsiveness. Our evaluation provides one of the first models of assessing fidelity to a blended, multi-phased implementation strategy and important insights into both the potential for ASD community-based agencies to use the toolkit effectively and implementation strategy fidelity more broadly. Overall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were relatively high, which supports the potential for the toolkit to be used with fidelity in ASD community agencies. Given that EBPs for ASD are often inconsistently or mis-implemented in community settings despite their potential to improve outcomes for a growing clinical population, understanding effective use of implementation strategies, such as the ACT SMART Toolkit, could contribute to reducing the EBP research-to-practice gap ( 20 – 24 ). Although we found fidelity to be high overall, there was some variability in implementation strategy fidelity by toolkit phase. Specifically, we found that dose was significantly lower in the preparation phase (Phase 3) compared to the implementation phase (Phase 4). One possible rationale for this finding is that there were substantial differences in demands for toolkit activities by phase. Indeed, the preparation phase required gathering materials, evaluating prospective adaptations, and developing training and adaptation plans whereas the implementation phase required carrying out and evaluating the developed plans. Indeed, there were both lower adherence and dose ratings for toolkit activities such as developing adaptation and implementation plans compared to toolkit activities related to evaluating treatments, funding, and training. Thus, the lower dose in the preparation phase may reflect the need to lower the amount or intensity of toolkit activities required to better align with ASD community agency’s capacity to plan for implementation. Considering recently identified context-specific barriers and facilitators to the ACT SMART Toolkit would also likely be critical to enhancing Phase 4 ( 32 , 55 ). Another potential rationale for significantly lower dose during the preparation phase compared to the implementation phase may be that ASD community agencies perceived greater value in implementing the chosen EBP than in planning for its implementation. While agency implementation teams were rated as moderately to very engaged during the preparation phase, it is unclear how well facilitators were able to emphasize the important relationship between planning and implementation. However, researchers have recently proposed that fostering this understanding is necessary to support successful and sustainable implementation ( 56 ). Thus, the ACT SMART Toolkit may also benefit from incorporating a greater focus on the practical importance of planning for implementation of EBPs. Implementation Strategy Fidelity Theory. Taken together, our instrumental case study assessment of fidelity to the ACT SMART Toolkit and exploration of the potential relationship between fidelity and EBP use within ASD community agencies notably provide one of the first models of assessing implementation strategy fidelity. Although a considerable amount of research has been conducted on intervention fidelity , few researchers have explored implementation strategy fidelity ( 35 , 49 , 50 ). For example, Slaughter et al. ( 35 ) found that no studies reporting on fidelity to implementation included a specific definition or theoretical framework for assessing implementation strategy fidelity. To our knowledge, only Berry and colleagues ( 49 ) recently adapted the Conceptual Framework for Implementation Fidelity to guide their evaluation of fidelity to practice facilitation as a strategy to improve primary care practices’ adoption of evidence-based guidelines for cardiovascular disease. Despite limited research, evaluating and understanding implementation strategy fidelity has important implications and is identified as a research priority within dissemination and implementation science ( 44 – 48 ). High fidelity to an implementation strategy may be reflective of other important implementation outcomes, such as high acceptability, appropriateness, and feasibility ( 57 , 58 ). Further, implementation strategy fidelity may inform determination of which components of a strategy are required to produce change and which can be removed or adapted to account for varying contextual characteristics ( 41 – 43 ). This knowledge may allow for demand optimization when the implementation strategy is being used, which may be particularly important when users of an implementation strategy have competing priorities or contextual factors that make completing the entirety of a blended implementation strategy difficult ( 32 ). From our instrumental case study of ACT SMART Toolkit fidelity, we have demonstrated that fidelity to blended, multi-phased implementation strategies is possible. Further, we have highlighted that implementation strategy fidelity may vary according to differing components of a strategy, such as components focusing on preparation for implementation versus components focusing on implementation itself. We also observed that implementation strategy fidelity may vary by context. Here, implementation strategy fidelity was observed to vary across different ASD community agencies using the ACT SMART Toolkit. These findings suggest that a next step to further understand implementation strategy fidelity may be investigating shifts across both strategy content and context. Importantly, increasing this understanding could then also inform commonly needed adaptations to improve implementation strategy fidelity. Strengths. We propose a main strength of our investigation is that we demonstrate one of the first instrumental case studies to consider fidelity to a blended, multi-phased implementation strategy. Importantly, our assessment of fidelity to the ACT SMART Toolkit may be able to provide a framework for other evaluations of implementation strategy fidelity and inform the underlying theory of implementation strategy fidelity. Within our evaluation, we also importantly found overall high fidelity to the toolkit within ASD community-based agencies and identified potential ways in which to optimize demands of the toolkit and increase sustainability. Limitations. In contrast, important limitations of our investigation include potential issues with measurement of specific implementation strategy fidelity variables. For example, Berry and colleagues ( 49 ) recently considered participant responsiveness as a moderator of implementation strategy fidelity rather than a component of fidelity itself, as it was considered in our analysis. Moreover, the potential issues with measurement may have been compounded by the fact that standard measures were not used for dose and participant responsiveness. However, as an emerging field, implementation science often faces issues related to measurement and standardized measures specific to implementation strategy fidelity have not yet been developed ( 46 , 47 , 59 ). Researchers have developed some standard measures for intervention fidelity, and these may be able to be adapted to assess implementation strategy fidelity in the future ( 60 ). Another potential limitation in our investigation is that there were different raters for adherence, dose, and participant responsiveness. While an independent observer rated adherence and dose for each implementation team, participant responsiveness was rated by a facilitator following implementation teams’ facilitation meetings. Although this presents potential for bias, direct observation by independent observers and even implementers have been found to be more accurate than collecting reports directly from participants ( 60 ). Further, when two facilitators independently gave ratings for participant responsiveness, there were high rates of agreement. Moreover, while we were generally able to assess implementation strategy fidelity by toolkit phase and activities, we were unable to assess all variables for all activities and by toolkit facet (i.e., web-based interface versus facilitation meetings). Thus, we are unable to make conclusions about all activities and the impact of the blended nature of the toolkit on implementation strategy fidelity. Further, our results may not generalize to discrete implementation strategies, which may benefit from their own instrumental case studies. Lastly, the most important limitation of our assessment of fidelity to the ACT SMART Toolkit was the limited sample size that rendered us under-powered to fully evaluate relationships between implementation strategy fidelity and EBP use. Moreover, our limited sample size also precluded us from considering additional factors such as implementation team and provider demographics and organizational climate within ASD community agencies. While we were able to observe variable implementation strategy fidelity across ASD community agencies, we were not yet able to identify consistent patterns related to higher or lower implementation strategy fidelity. However, there is evidence that some of these factors may moderate the relationship between implementation strategy fidelity to the ACT SMART toolkit and increased EBP use ( 61 ). Future research would benefit from consideration of potential moderators of implementation strategy fidelity and utilizing standard measures and independent raters ( 59 – 64 ). In addition, future studies may benefit from a design intended to systematically evaluate fidelity to all components of a strategy. These lines of research may provide further insight into both effective use of the ACT SMART Toolkit as well as the advancing the field of implementation science more broadly. Conclusions By utilizing an instrumental case study approach, we advanced understanding of effective use of the ACT SMART Toolkit as well as the theory of implementation strategy fidelity more broadly. We found that the ACT SMART Toolkit has potential to be used with high fidelity in ASD community-based agencies. However, we also found that there was some variability in fidelity among toolkit phases, which points to possible adaptations needed to improve toolkit use even further. Considering adaptations may be critical as these findings may reflect that fidelity to blended, multi-phased implementation strategies is dynamic and affected by both strategy content and context. By increasing the use of and fidelity to effective implementation strategies that facilitate EBP adoption, utilization and sustainment within community-based settings, there is potential to increase overall public health. Abbreviations ASD Autism Spectrum Disorder EBP Evidence-Based Practice ACT SMART Toolkit Autism Community Toolkit:Systems to Measure and Adopt Research-Based Treatments EPIS model Exploration, Preparation, Implementation, Sustainment model ABA Applied Behavior Analysis Declarations Ethics approval and consent participate : All participants provided informed consent. The present study received exempt status from IRB review from (MASKED) to conduct secondary analysis. Data was collected while the senior author was at (MASKED), under the IRB approval # (MASKED). Consent for publication : Not applicable Availability of data and materials: The datasets used and/or analyzed during the current study are available from the senior author (MASKED) upon reasonable request. Competing interests: The authors declare that they have no competing interests. Funding: This work was supported by (MASKED). Funding was used for the design and data collection of the pilot study of the ACT SMART toolkit. Authors’ contributions: (First Author) contributed to conceptualization, selection of methodology, formal analysis, and writing of the original draft of the present manuscript. (Senior Author) conducted the pilot study of the ACT SMART Toolkit and contributed to conceptualization, selection of methodology, and review and editing of the present manuscript. Both authors read and approved the final manuscript. Acknowledgments: (Senior Author) would like to express gratitude to the (MASKED), who provided valuable insight and recommendations as the Toolkit was developed. Additionally, (Senior Author) would like to thank Dr. Patricia Chamberlain for permitting the adaptation of the Stages of Implementation Completion. (First Author) and (Senior Author) would also like to acknowledge (MASKED), who provided invaluable statistical consultation. References Blaxill M, Rogers T, Nevison C. Autism Tsunami: The Impact of Rising Prevalence on the Societal Cost of Autism in the United States. J Autism Dev Disord. 2022; 52(6):2627–43. doi:10.1007/s10803-021-05120-7 Leigh JP, Du J. Brief Report: Forecasting the Economic Burden of Autism in 2015 and 2025 in the United States. J Autism Dev Disord. 2015; 45(12):4135–9. doi:10.1007/s10803-015-2521-7 Maenner MJ, Shaw KA, Bakian AV, Bilder DA, Durkin MS, Esler A, et al. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. MMWR Surveill Summ. 2021;70(11):1–16. doi:10.15585/mmwr.ss7011a1 American Psychiatric Association. Autism spectrum disorder. In: Diagnostic and statistical manual of mental disorders. 5th ed. Arlington, VA: American Psychiatric Association; 2013. doi:10.1176/appi.books.9780890425596 Lai MC, Lombardo MV, Baron-Cohen S. Autism. The Lancet. 2014;383(9920):896–910. doi:10.1016/S0140-6736(13)61539-1 Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric Disorders in Children With Autism Spectrum Disorders: Prevalence, Comorbidity, and Associated Factors in a Population-Derived Sample. Journal of the American Academy of Child & Adolescent Psychiatry. 2008;47(8):921–9. doi:10.1097/CHI.0b013e318179964f Hattier MA, Matson JL, Belva BC, Horovitz M. The occurrence of challenging behaviours in children with autism spectrum disorders and atypical development. Developmental Neurorehabilitation. 2011;14(4):221–9. doi: 10.3109/17518423.2011.573836 Horner RH, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: A research synthesis. Journal of Autism and Developmental Disorders. 2002;32(5):423–46. doi:10.1023/A:1020593922901 Stevens E, Atchison A, Stevens L, Hong E, Granpeesheh D, Dixon D, Linstead E. A cluster analysis of challenging behaviors in autism spectrum disorder. In 2017 16 th IEEE International Conference on Machine Learning and Applications (ICMLA); 2017 December; IEEE. p.661-6. Kim SY, Bottema-Beutel K. A meta regression analysis of quality of life correlates in adults with ASD. Research in Autism Spectrum Disorders. 2019;63:23–33. doi: 10.1016/j.rasd.2018.11.004 Mason D, Mackintosh J, McConachie H, Rodgers J, Finch T, Parr JR. Quality of life for older autistic people: The impact of mental health difficulties. Research in Autism Spectrum Disorders. 2019;63:13–22. doi:10.1016/j.rasd.2019.02.007 Bottema-Beutel K, Kapp SK, Lester JN, Sasson NJ, Hand BN. Avoiding Ableist Language: Suggestions for Autism Researchers. Autism in Adulthood. 2021;3(1):18–29. doi:10.1089/aut.2020.0014 Pitney J. Lifetime social cost. Autism Politics and Policy [Internet]. Feb 6, 2020. Available from http://www.autismpolicyblog.com/2020/02/lifetime-social-cost.html Robertson SM. Neurodiversity, Quality of Life, and Autistic Adults: Shifting Research and Professional Focuses onto Real-Life Challenges. DSQ. 2010;30(1). doi: 10.18061/dsq.v30i1.1069 King M, Bearman P. Diagnostic change and the increased prevalence of autism. International Journal of Epidemiology. 2009;38(5):1224–34. doi:10.1093/ije/dyp261 Eapen V, Črnčec R, Walter A. Clinical outcomes of an early intervention program for preschool children with Autism Spectrum Disorder in a community group setting. BMC Pediatr. 2013;13(1):3. doi:10.1186/1471-2431-13-310.1186/1471-2431-13-3 Horlin C, Falkmer M, Parsons R, Albrecht MA, Falkmer T. The Cost of Autism Spectrum Disorders. Mulle JG, editor. PLoS ONE. 2014;9(9):e106552. doi:10.1371/journal.pone.0106552 Vinen Z, Clark M, Paynter J, Dissanayake C. School Age Outcomes of Children with Autism Spectrum Disorder Who Received Community-Based Early Interventions. J Autism Dev Disord. 2018;48(5):1673–83. doi:10.1007/s10803-017-3414-8 Elder JH, Brasher S, Alexander B. Identifying the Barriers to Early Diagnosis and Treatment in Underserved Individuals with Autism Spectrum Disorders (ASD) and Their Families: A Qualitative Study. Issues in Mental Health Nursing. 2016;37(6):412–20. doi: 10.3109/01612840.2016.1153174 Drahota A, Meza RD, Bustos TE, Sridhar A, Martinez JI, Brikho B, et al. Implementation-as-Usual in Community-Based Organizations Providing Specialized Services to Individuals with Autism Spectrum Disorder: A Mixed Methods Study. Adm Policy Ment Health. 2021;48(3):482–98. doi:10.1007/s10488-020-01084-5 Paynter JM, Ferguson S, Fordyce K, Joosten A, Paku S, Stephens M, et al. Utilisation of evidence-based practices by ASD early intervention service providers. Autism. 2017;21(2):167–80. doi:10.1177/1362361316633032 Pickard K, Meza R, Drahota A, Brikho B. They’re Doing What? A Brief Paper on Service Use and Attitudes in ASD Community-Based Agencies. Journal of Mental Health Research in Intellectual Disabilities. 2018;11(2):111–23. doi:10.1080/19315864.2017.1408725 Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, et al. Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder: A Comprehensive Review. J Autism Dev Disord. 2015;45(7):1951–66. doi:10.1007/s10803-014-2351-z Wood JJ, McLeod BD, Klebanoff S, Brookman-Frazee L. Toward the Implementation of Evidence-Based Interventions for Youth With Autism Spectrum Disorders in Schools and Community Agencies. Behavior Therapy. 2015;46(1):83–95. doi:10.1016/j.beth.2014.07.003 Drahota A, Chlebowski C, Stadnick N, Baker-Ericzén MJ, Brookman-Frazee L. The dissemination and implementation of behavioral treatments for anxiety in ASD. In: Kerns C, Renno P, Storch A, Kendall PC, Wood JJ, editors. Anxiety in children and adolescents with autism spectrum disorder: Evidence-based assessment and treatment. Atlanta, GA: Elsevier; 2017. p.231-249. Drahota A, Meza R, Martinez JI. The autism-community toolkit: Systems to measure and adopt research-based treatments. 2014. Available from www.actsmartoolkit.com Gomez E, Drahota A, Stahmer AC. Choosing strategies that work from the start: A mixed methods study to understand effective development of community–academic partnerships. Action Research. 2021;19(2):277–300. doi:10.1177/1476750318775796 Drahota A, Aarons GA, Stahmer AC. Developing the Autism Model of Implementation for Autism spectrum disorder community providers: study protocol. Implementation Sci. 2012;7(1):85. doi:10.1186/1748-5908-7-85 Aarons GA, Hurlburt M, Horwitz SM. Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Adm Policy Ment Health. 2011;38(1):4–23. doi:10.1007/s10488-010-0327-7 Drahota A., Meza R., Martinez JI, Sridhar A., Bustos TE, Tschida J., Stahmer A, Aarons, GA. Feasibility, Acceptability, and Utility of the ACT SMART Implementation Toolkit. In preparation. Sridhar A, Drahota A. Preliminary effectiveness of the ACT SMART implementation toolkit: facilitating evidence-based practice implementation in community-based autism organizations. International Journal of Developmental Disabilities. 2022:1-6. doi:10.1080/20473869.2022.2065448 Sridhar A, Drahota A, Walsworth K. Facilitators and barriers to the utilization of the ACT SMART Implementation Toolkit in community-based organizations: a qualitative study. Implement Sci Commun. 2021;2(1):55. doi:10.1186/s43058-021-00158-1 Allen JD, Shelton RC, Emmons KM, Linnan LA (2018). Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Brownson RC, Colditz GA, Proctor, EK, editors. Dissemination and implementation research in health: Translating science to practice. Oxford Scholarship Online; 2017. https://doi.org/10.1093/oso/9780190683214.003.0016 Mowbray CT, Holter MC, Teague GB, Bybee D. Fidelity Criteria: Development, Measurement, and Validation. American Journal of Evaluation. 2003;24(3):315–40. doi:10.1177/109821400302400303 Slaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implementation Sci. 2015;10(1):129. doi:10.1186/s13012-015-0320-3 Dusenbury L. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research. 2003;18(2):237–56. doi:10.1093/her/18.2.237 Teague GB. Fidelity. Implementation Research Institute Presentation; 2013. Century J, Rudnick M, Freeman C. A Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge. American Journal of Evaluation. 2010;31(2):199–218. doi:10.1177/1098214010366173 Metz A, Bartley L. Implementation teams: A stakeholder view of leading and sustaining change. In: Implementation Science 3.0. Springer, Cham; 2020. p.199-225. Cross W, West J. Examining implementer fidelity: conceptualising and measuring adherence and competence. Journal of Children’s Services. 2011;6(1):18–33. doi:10.5042/jcs.2011.0123 Kirk MA, Haines ER, Rokoske FS, Powell BJ, Weinberger M, Hanson LC, et al. A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Translational Behavioral Medicine. 2021;11(1):21–33. doi: 10.1093/tbm/ibz178 Mihalic S. The importance of implementation fidelity. Emotional and Behavioral Disorders in Youth . 2004;4(4):83-105. http://www.incredibleyears.com/wp-content/uploads/fidelity-importance.pdf Perez Jolles M, Lengnick-Hall R, Mittman BS. Core Functions and Forms of Complex Health Interventions: a Patient-Centered Medical Home Illustration. J GEN INTERN MED. 2019;34(6):1032–8. doi:10.1007/s11606-018-4818-7 The CIPHER team, Haynes A, Brennan S, Redman S, Williamson A, Gallego G, et al. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implementation Sci. 2015;11(1):23. doi:10.1186/s13012-016-0378-6 Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258–h1258. doi:10.1136/bmj.h1258 Akiba CF, Powell BJ, Pence BW, Nguyen MXB, Golin C, Go V. The case for prioritizing implementation strategy fidelity measurement: benefits and challenges. Translational Behavioral Medicine. 2022;12(2):335–42. doi:10.1093/tbm/ibab138 Akiba CF, Powell BJ, Pence BW, Muessig K, Golin CE, Go V. “We start where we are”: A qualitative study of barriers and pragmatic solutions to the assessment and reporting of implementation strategy fidelity [Internet]. In Review; 2022 [cited 2022 Jul 13]. Available from: https://www.researchsquare.com/article/rs-1626073/v1 Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health. 2019;7:3. doi:10.3389/fpubh.2019.000031. Berry CA, Nguyen AM, Cuthel AM, Cleland CM, Siman N, Pham-Singer H, et al. Measuring Implementation Strategy Fidelity in HealthyHearts NYC: A Complex Intervention Using Practice Facilitation in Primary Care. American Journal of Medical Quality. 2021;36(4):270–6. doi:10.1177/1062860620959450. Kourouche S, Curtis K, Munroe B, Watts M, Balzer S, Buckley T. Implementation strategy fidelity evaluation for a multidisciplinary Chest Injury Protocol (ChIP). Implement Sci Commun. 2021;2(1):86. doi:10.1186/s43058-021-00189-8 Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100. doi:10.1186/1471-2288-11-100 Chamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: The Stages of implementation completion (SIC). Implementation Sci. 2011;6(1):116. doi:10.1186/1748-5908-6-116 Drahota A, Martinez JI. ACT SMART Milestones and Activity Fidelity Forms. Unpublished measure. 2014. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. 2017;356:i6795. Powell BJ, Haley AD, Patel SV, Amaya-Jackson L, Glienke B, Blythe M, et al. Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). Implement Sci Commun. 2020;1(1):9. doi:10.1186/s43058-020-00009-5 Leal Filho W, Skanavis C, Kounani A, Brandli LL, Shiel C, Paço A do, et al. The role of planning in implementing sustainable development in a higher education context. Journal of Cleaner Production. 2019;235:678–87. doi:10.1016/j.jclepro.2019.06.322 Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38(2):65–76. doi:10.1007/s10488-010-0319-7 Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implementation Sci. 2017;12(1):108. doi:10.1186/s13012-017-0635-3 Lewis CC, Dorsey C. Advancing implementation science measurement. In: Albers B, Shlonsky A, Mildon R, editors. Implementation Science 3.0. Springer Nature; 2020. p.227. Ibrahim S, Sidani S. Fidelity of Intervention Implementation: A Review of Instruments. Health. 2015;07(12):1687–95. doi:10.4236/health.2015.712183 Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Sci. 2012;7(1):23. doi:10.1186/1748-5908-7-23 Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase ME, Weiss RD, et al. The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–40. doi:10.1080/10503300500288951 Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology. 2008;76(4):544–55. doi:10.1037/0022-006X.76.4.544 McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, et al. Fidelity Outcomes in the National Implementing Evidence-Based Practices Project. PS. 2007;58(10):1279–84. doi:10.1176/ps.2007.58.10.1279 Supplementary Files StaRIchecklistforauthorcompletionFinal.pdf Cite Share Download PDF Status: Published Journal Publication published 16 May, 2023 Read the published version in Implementation Science Communications → Version 1 posted Reviewers agreed at journal 25 Sep, 2022 Reviewers invited by journal 24 Sep, 2022 Editor assigned by journal 05 Aug, 2022 Submission checks completed at journal 04 Aug, 2022 Editor invited by journal 04 Aug, 2022 First submitted to journal 04 Aug, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1930423","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research","associatedPublications":[],"authors":[{"id":126929684,"identity":"f9b478f2-ed37-4e69-945d-08f60b7c4d16","order_by":0,"name":"Jessica Tschida","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAv0lEQVRIiWNgGAWjYHACNoYENgZmfgiHmbB6HpgWyQaStICQwQFitdjzH3724EGZHbvxtePPJBgqrBMbCNoikWZukHAumdnsdo6ZBMOZdGK08LBJJLYxg7SwSTC2HSZCC/8ZkJZ6ZuPZ6c8kGP8Ro4UhB6TlMLOBdIKZBGMDMVpupJlJJJw7zixxO8fYIuFYujFBLez9h59J/iirTuafnf7wxocaa1mCWmAgGUwmEKscBOxIUTwKRsEoGAUjDAAAjHk1z/NvaOUAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0002-3701-417X","institution":"Michigan State University Department of Psychology","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Jessica","middleName":"","lastName":"Tschida","suffix":""},{"id":126929685,"identity":"b756c5b1-d30e-472b-b862-5b7c500547ff","order_by":1,"name":"Amy Drahota","email":"","orcid":"","institution":"Michigan State University Department of Psychology","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Amy","middleName":"","lastName":"Drahota","suffix":""}],"badges":[],"createdAt":"2022-08-04 16:52:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1930423/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1930423/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s43058-023-00434-2","type":"published","date":"2023-05-16T20:51:11+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":24937923,"identity":"f66ce3eb-3ba4-45bf-b2d8-b7058c0648bf","added_by":"auto","created_at":"2022-08-08 17:59:58","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":39064,"visible":true,"origin":"","legend":"\u003cp\u003eAdapted EPIS Framework with ACT SMART Implementation Toolkit Steps and Activities to Support EBP Implementation\u0026nbsp;\u003c/p\u003e","description":"","filename":"Figure1ACTSMARTFidelityFinal.png","url":"https://assets-eu.researchsquare.com/files/rs-1930423/v1/24ff0731d18fa69010d3cc77.png"},{"id":44729590,"identity":"1d436242-0a29-45a9-bda5-a53ff1714a3b","added_by":"auto","created_at":"2023-10-16 21:19:08","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":905236,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1930423/v1/3061de48-28df-41d2-8fe3-b7be5b852276.pdf"},{"id":24937924,"identity":"9a9561bb-bc78-4651-bf0c-39359b882eef","added_by":"auto","created_at":"2022-08-08 17:59:59","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":229141,"visible":true,"origin":"","legend":"","description":"","filename":"StaRIchecklistforauthorcompletionFinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1930423/v1/1cb1d2b1444cecb517fd705f.pdf"}],"financialInterests":"","formattedTitle":"Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity","fulltext":[{"header":"Contributions to the literature","content":"\u003cul\u003e\n \u003cli\u003eAssessing implementation strategy fidelity is critical to advance the field of implementation science but is rarely evaluated in extant literature.\u003c/li\u003e\n \u003cli\u003eIn an instrumental case study, we found high but variable fidelity to a blended, multi-phased implementation strategy (the ACT SMART Toolkit) for increasing EBP use in ASD community agencies.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eThese findings provide evidence that community-based providers utilize the ACT SMART Toolkit with fidelity, furthering the rationale that the toolkit is a promising strategy to increase EBP use within ASD community agencies.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eThe paper contributes an innovative model of assessing implementation strategy fidelity, suggesting important influences of strategy content and context.\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"Background","content":"\u003cp\u003e\u003cstrong\u003eAutism Spectrum Disorder.\u003c/strong\u003e An autism spectrum disorder (ASD) affects approximately 1 in 44 children in the United States and has been identified as a public health concern estimated to cost $461\u0026nbsp;billion dollars a year for services and treatment by 2030 (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e). ASD is characterized by social and communication difficulties as well as restricted and repetitive behaviors and interests, and commonly co-occurs with anxiety disorders, obsessive compulsive disorder, attention deficit hyperactivity disorder, and/or oppositional defiant disorder (\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e6\u003c/span\u003e). Additionally, children on the autism spectrum have higher rates of behaviors such as self-injury, aggression, tantrums, and property destruction compared to neurotypical peers (\u003cspan class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eBoth the core features and co-occurring disorders and behaviors of ASD have been found to predict unsatisfactory outcomes in quality-of-life factors, including peer relationships, educational attainment, employment, and independent living in adulthood (\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e). Associations between autistic\u003ca id=\"#FNLinkFn1\" class=\"FNLink\" href=\"#Fn1\"\u003e\u003c/a\u003e characteristics and unsatisfactory quality-of-life outcomes are also maintained by systemic barriers to inclusion of autistic individuals, such as societal stigma and lack of appropriate accommodations, in education, employment, and housing opportunities (\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eThe prevalence rate for ASD continues to grow dramatically as practices for diagnosis improve (\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e15\u003c/span\u003e). However, despite their potential to improve outcomes for autistic youth and reduce individual and societal costs (\u003cspan class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e), barriers to community level identification and intervention remain (\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e19\u003c/span\u003e). Although evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children, they are often inconsistently implemented or mis-implemented in community-based settings where many autistic children receive services (\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e24\u003c/span\u003e). As a result, there is a considerable number of children on the autism spectrum not receiving practices empirically demonstrated to improve outcomes as part of their usual care. Thus, there is a need to identify, develop, and evaluate strategies facilitating the implementation of EBPs for ASD within community settings.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eACT SMART Implementation Toolkit.\u003c/strong\u003e Drahota and colleagues (\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e) developed a blended implementation strategy to support the implementation of EBPs for ASD in community settings: Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments (ACT SMART Toolkit). The ACT SMART Toolkit was developed through a review of existing implementation strategy evidence and by incorporating insight from a community-academic partnership (\u003cspan class=\"CitationRef\"\u003e27\u003c/span\u003e). The ACT SMART Toolkit involves facilitation meetings led by trained ACT SMART facilitators and a web-based interface to guide ASD community agency leaders, supervisors, and providers that comprise agency implementation teams through phases of implementing an EBP (\u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e28\u003c/span\u003e). Drahota and colleagues (\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e25\u003c/span\u003e) designed the ACT SMART Toolkit to have steps and activities that align with an adapted implementation model, the adapted Exploration, Adoption, Preparation, Implementation, Sustainment (EPIS) model (\u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e29\u003c/span\u003e). Overall, the toolkit guides ASD agency implementation teams to explore their agency\u0026rsquo;s receptivity to implementing a new EBP, identify and decide upon an EBP that meets their needs, prospectively plan to implement the EBP, implement the EBP, and finally evaluate implementation and develop a plan for sustainment (See Fig.\u0026nbsp;1; 25).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eImportantly, the ACT SMART Toolkit has been pilot tested with six ASD community-based agencies. Preliminary work by Drahota and colleagues (in preparation) suggests that the toolkit is feasible, acceptable, and useful to agency implementation teams (\u003cspan class=\"CitationRef\"\u003e30\u003c/span\u003e). In addition, Sridhar and Drahota (\u003cspan class=\"CitationRef\"\u003e31\u003c/span\u003e) found that the toolkit facilitates clinically meaningful changes in agency provider- and supervisor-reported EBP use. Moreover, Sridhar and colleagues (\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e) identified salient facilitators (i.e., facilitation, facilitation meetings, and phase specific activities) and salient barriers (i.e., website issues, perceived lack of resources, and contextual factors within ASD community agencies such as time constraints and funding) to the utilization of the ACT SMART Toolkit in the pilot study. Therefore, the next incremental, yet crucial, step in evaluating initial use of the ACT SMART Toolkit is to assess implementation strategy fidelity.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eImplementation Strategy Fidelity.\u003c/strong\u003e Fidelity is a construct that assesses the extent to which individuals (e.g., providers) deliver a strategy as planned (\u003cspan class=\"CitationRef\"\u003e33\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e). Researchers have proposed components that contribute to fidelity include: (\u003cspan class=\"CitationRef\"\u003e1\u003c/span\u003e) adherence to the outlined procedures, (\u003cspan class=\"CitationRef\"\u003e2\u003c/span\u003e) proportion of the strategy received (i.e., dose), (\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e) extent of individual responsivity to the strategy (i.e., participant responsiveness), (\u003cspan class=\"CitationRef\"\u003e4\u003c/span\u003e) quality of implementation, and (\u003cspan class=\"CitationRef\"\u003e5\u003c/span\u003e) differentiation from unspecified procedures (\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e). Researchers have also proposed that quality and differentiation primarily capture the characteristics of an EBP being implemented whereas adherence, dose, and participant responsiveness hold relevance for implementation strategy fidelity (\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eDusenbury (\u003cspan class=\"CitationRef\"\u003e36\u003c/span\u003e) defines \u003cem\u003eadherence\u003c/em\u003e as the extent to which activities are consistent with the way a strategy is proposed, \u003cem\u003edose\u003c/em\u003e as the amount of strategy content received by participants, and \u003cem\u003eparticipant responsiveness\u003c/em\u003e as the extent to which participants are engaged by and involved in the strategy. In relation to the ACT SMART Toolkit, participants would refer to the agency implementation teams (i.e., a group of individuals within an agency responsible for facilitating EBP implementation; 39).\u003c/p\u003e\n\u003cp\u003eFidelity is also considered dynamic and may be influenced by factors such as provider characteristics, the setting, and/or complexity of the strategy (\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e40\u003c/span\u003e). Assessing implementation strategy fidelity can help implementation strategy developers further understand which components of an implementation strategy may be core functions needed to produce desired outcomes and which components may be adapted to account for varying contextual characteristics (\u003cspan class=\"CitationRef\"\u003e41\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e43\u003c/span\u003e). Of course, this is contingent upon an ability to determine whether implementation of the strategy remained consistent with its underlying theory (\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e45\u003c/span\u003e). Notably, increasing understanding about how implementation strategies work has been identified as a research priority within the field of dissemination and implementation science (\u003cspan class=\"CitationRef\"\u003e46\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e48\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eDespite its importance, fidelity to implementation strategies has rarely been assessed; instead, research has often focused only on fidelity to the EBPs being implemented (\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e). Indeed, Slaughter et al. (\u003cspan class=\"CitationRef\"\u003e35\u003c/span\u003e) conducted a scoping review that indicated no articles reporting fidelity to implementation strategies included definitions or conceptual frameworks for assessing implementation strategy fidelity. To our knowledge, few studies have examined fidelity to an implementation strategy, and only one recent study has used a guiding theoretical framework (\u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e50\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePresent Study.\u003c/strong\u003e Using an instrumental case study approach to assess fidelity to the ACT SMART toolkit during its pilot study may be able to provide important insights into the use of the toolkit as well as the phenomenon of implementation strategy fidelity more broadly (\u003cspan class=\"CitationRef\"\u003e51\u003c/span\u003e). Examining implementation strategy fidelity can provide insight into the overall potential for ASD community-based agencies to use the toolkit as planned and ultimately report greater use of EBPs. This information may be particularly useful for ASD community-based agencies, given potential competing priorities and contextual barriers to completing the toolkit in its entirety (\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e). Further, we provide one of the first process models to assess fidelity to a blended, multi-phased implementation strategy. This model may then inform a broader understanding of implementation strategy fidelity and contribute to underlying theory.\u003c/p\u003e\n\u003cp\u003eSpecifically, we addressed two key questions:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003eWhat was fidelity to the ACT SMART Toolkit at an aggregate and individual agency level according to adherence, dose, and participant responsiveness during its pilot study?\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eWas fidelity to the ACT SMART Toolkit significantly different by toolkit phases?\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Methods","content":"\u003cp\u003e \u003cb\u003eParticipants.\u003c/b\u003e A total of six ASD community agencies located in Southern California were included in the pilot study of the ACT SMART toolkit. Four of the ASD community agencies were Applied Behavior Analysis (ABA) organizations, one was an ABA and mental health organization, and one agency was a Speech and Language Pathology organization. Five of the six ASD community agencies chose to adopt the EBP, Video Modeling, during the pilot study and complete all phases of the ACT SMART toolkit. One ABA agency chose not to adopt an EBP at the end of the adoption decision phase of the toolkit.\u003c/p\u003e \u003cp\u003eEach ASD community agency developed implementation teams composed of agency staff (see Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e for implementation team demographic and discipline information). At least one agency leader was required for each implementation team. Eligibility criteria for agency leaders were: (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) holding the role of CEO, director, or leading decision-maker regarding treatment use at an ASD community agency eligible to participate in the ACT SMART pilot study, (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) willingness to participate in the pilot study for 1 year, and (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) agreement to provide feedback after completing each phase of the pilot study. The agency leader for each participating agency then invited up to four other agency staff members (i.e., supervisors and direct providers) to complete their agency\u0026rsquo;s implementation team. Eligibility criteria for implementation team members was to agree to complete the toolkit and provide feedback about its feasibility, acceptability, and utility.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDemographic and discipline information across implementation teams\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAgency Leaders\u003c/p\u003e \u003cp\u003e(\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;7)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSupervisors\u003c/p\u003e \u003cp\u003e(\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;8)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eDirect Providers\u003c/p\u003e \u003cp\u003e(\u003cem\u003en\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eSex Assigned at Birth (Females)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eRace\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eWhite\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMixed Race\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePrefer Not to Answer\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12.5%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMissing\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e37%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eEducation Level\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMaster\u0026rsquo;s Degree\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e42.9%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e50%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDoctorate\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e57.1%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12.5%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMissing\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e37%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eDiscipline\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePsychology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e28.6%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBehavior Specialist\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e28.6%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e25%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSpeech/Language/Communication\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e28.6%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e12.5%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEducation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e14.3%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMissing\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e37%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eMaterials \u0026amp; Procedure.\u003c/b\u003e As part of the pilot study, a research assistant served as an independent observer and evaluated implementation teams\u0026rsquo; fidelity using the Implementation Milestones form, adapted with permission from the Stages of Implementation Completion (\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e), and the ACT SMART Activity Fidelity form (\u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e). The ACT SMART Implementation Milestones form required the independent observer to record a Yes or No answer (scored as 1 and 0, respectively) for whether activities during pre-implementation and phase 1 through phase 4 of the ACT SMART Toolkit were completed. Scores were converted into percentages to assist with interpretation. The ACT SMART Activity Fidelity form presented more detailed questions regarding completion of activities during Phase 2: Adoption; Phase 3: Preparation; and Phase 4: Implementation. The independent observer recorded a Yes or No answer (scored as 1 and 0, respectively) for whether implementation teams completed each activity and then rated the amount of the form completed using a 4-point Likert scale where 0 = \u0026ldquo;Nothing Completed\u0026rdquo;, 1 = \u0026ldquo;Minimally Completed (1\u0026ndash;2 items)\u0026rdquo;, 2 = \u0026ldquo;Moderately Completed (3\u0026ndash;4 items)\u0026rdquo;, and 3 = \u0026ldquo;Mostly/All Completed (5\u0026ndash;6 items)\u0026rdquo;.\u003c/p\u003e \u003cp\u003eIn addition to the observational data collected using the ACT SMART Implementation Milestones form and the ACT SMART Activity Fidelity form, ACT SMART facilitators rated implementation team engagement using the ACT SMART Implementation Team Engagement Rating Scale that was created by the toolkit developers. Immediately after each facilitation meeting, the ACT SMART facilitator(s) rated implementation team engagement in ACT SMART activities and facilitation meetings since the last facilitation meeting occurred. Engagement ratings were completed using a 5-point Likert scale where 1 = \u0026ldquo;Not at all engaged\u0026rdquo;, 2 = \u0026ldquo;Slightly Engaged\u0026rdquo;, 3 = \u0026ldquo;Moderately Engaged\u0026rdquo;, 4 = \u0026ldquo;Very Engaged\u0026rdquo;, and 5 = \u0026ldquo;Extremely Engaged\u0026rdquo;.\u003c/p\u003e \u003cp\u003eIn the present study, we used the operational definitions from Dusenbury (\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e) and an overall scoring rubric for implementation strategy fidelity developed in Slaughter et al. (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e) as the basis for using the ACT SMART Implementation Milestones form, ACT SMART Activity Fidelity form, and ACT SMART Implementation Team Engagement Rating Scale to assess implementation strategy fidelity via adherence, dose, and participant responsiveness, respectively.\u003c/p\u003e \u003cp\u003e\u003cb\u003eAnalysis Plan.\u003c/b\u003e We used an instrumental case study approach to explore both fidelity to the ACT SMART Toolkit and potential generalizations to a broader underlying theory of implementation strategy fidelity. The Standards for Reporting Implementation Studies (StaRI) checklist was used to assist reporting given that the ACT SMART Toolkit is an implementation strategy developed to increase EBP use in ASD community agencies (\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e). First, we assessed adherence, dose, and participant responsiveness for the ACT SMART Toolkit overall as well as for each phase and activity of the toolkit. Utilizing the ACT SMART Implementation Milestones form, we assessed adherence via a Yes/No answer to whether implementation milestones were completed. Overall, by phase, and by activity, we calculated the average percentage of \u0026ldquo;Yes\u0026rdquo; answers for required toolkit activities. We assessed dose by analyzing Likert scales on the ACT SMART Activity Fidelity form evaluating how much of each activity was completed. Overall, by phase, and by activity, we calculated the mean dose rating. Finally, we assessed participant responsiveness by analyzing the Likert scales on the ACT SMART Implementation Team Engagement Rating Scale and used dates of completion to confirm phase. Overall and by phase, we calculated the mean participant responsiveness rating. We did not calculate the mean participant responsiveness rating by activity as ratings for engagement were only given by phase. We also calculated an average percent agreement on participant responsiveness ratings from facilitation meetings in which multiple facilitators were present. Lastly, we calculated overall, phase, and activity adherence, dose, and participant responsiveness for each agency implementation team.\u003c/p\u003e \u003cp\u003eTo evaluate whether adherence, dose, or participant responsiveness significantly differed by toolkit phase, we conducted repeated measures ANOVAs with toolkit phase as a within-subjects factor. We also conducted Bonferroni post-hoc tests and calculated effect sizes using local error terms. It should be noted that dose was not observed during phase 1 of the toolkit. Further, the one ASD community agency that chose not to adopt an EBP at the end of the adoption decision phase (Phase 2) of the toolkit did not have any implementation strategy fidelity variables observed during Phase 3 or Phase 4 of the toolkit.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e \u003cb\u003eAggregate Fidelity to the ACT SMART Toolkit.\u003c/b\u003e Agency implementation teams adhered to an overall average of 90% (\u003cem\u003eSD\u0026thinsp;=\u003c/em\u003e\u0026thinsp;11.3%) of required ACT SMART Toolkit activities. Average adherence ranged from 74% (\u003cem\u003eSD\u0026thinsp;=\u003c/em\u003e\u0026thinsp;19.5%) completion of required toolkit activities during the preparation phase of the toolkit to 100% (\u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0%) completion of required toolkit activities during the implementation phase of the toolkit (see Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). While completion rate for individual activities within phases was also relatively high across agencies, there was some variability. There were lower average completion rates for activities such as the benefit-cost estimator, gathering treatment materials, and developing adaptation and implementation plans compared to higher average completion rates for activities related to treatment evaluation, funding, and training.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eAdherence to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAggregate\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAgency 1\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAgency 2\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAgency 3\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eAgency 4\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAgency 5\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eAgency 6\u003c/p\u003e \u003cp\u003e\u003cem\u003eM%\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eOverall Adherence Scores\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e90% (11.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e90.8% (14.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e93.3% (14.9)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e91.7% (14.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e89.5% (17.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e88.0% (26.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e85.3% (20.2)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePre-Implementation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAgency first contacted\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAgency interest indicated\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAgency recruitment meeting\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eOrientation meeting attendance\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 1: Exploration\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e83% (18.0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e66.7% (57.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e66.7% (57.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e66.7% (57.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eRecruit for agency assessment\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e83% (40.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAgency assessment link sent\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eStaff response rate\u0026thinsp;\u0026ge;\u0026thinsp;75%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e67% (51.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 2: Adoption\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e92% (17.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e87.5% (35.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e75.0% (46.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e87.5% (35.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTreatment selection\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate fit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate feasibility\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate clinical utility and validity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e83% (40.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate training requirements\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate funding source\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate benefits and costs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e50% (54.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eValidate adoption decision\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e100\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 3: Preparation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e74% (19.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e60% (54.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e40% (54.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e60% (54.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGather and review treatment materials\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e60% (54.8)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate prospective adaptations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e80% (44.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop adaptation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e50% (70.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop training plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop implementation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e80% (44.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 4: Implementation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e100%\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCarry out adaptation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCarry out training plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eCarry out implementation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e100%\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e100\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e100\u003csup\u003e\u003cem\u003eb\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003cem\u003eNote.\u003c/em\u003e Adherence scoring range is 0\u0026ndash;100%.\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e Agency implementation team made the decision to not adopt an EBP.\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003csup\u003e\u003cem\u003eb\u003c/em\u003e\u003c/sup\u003e Agency implementation team created and carried out an implementation plan in Phase 4.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eRelated to dose, the independent observer gave agency implementation teams an overall average rate falling between \u0026ldquo;Moderately Completed\u0026rdquo; to \u0026ldquo;Mostly/All Completed\u0026rdquo; (\u003cem\u003eM\u0026thinsp;=\u003c/em\u003e\u0026thinsp;2.3, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.60). The lowest average dose rating was between \u0026ldquo;Minimally Completed\u0026rdquo; to \u0026ldquo;Moderately Completed\u0026rdquo; (\u003cem\u003eM\u0026thinsp;=\u003c/em\u003e\u0026thinsp;1.7, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.60) during the preparation phase whereas the highest average dose rating was between \u0026ldquo;Moderately Completed\u0026rdquo; to \u0026ldquo;Mostly/All Completed\u0026rdquo; (\u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;2.9, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.01) during the implementation phase of the toolkit (see Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Consistent with observations of adherence, there were lower average dose ratings for activities such as the benefit-cost estimator, gathering treatment materials, and developing adaptation and implementation plans compared to higher average completion rates for activities related to treatment evaluation, funding, training, and carrying out developed plans. Here, it should be noted that average dose ratings by activity could not be calculated for the implementation phase given that evaluation surveys during this phase were designed to be dynamic and capture completion of different sets of tasks by agency (\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eDose to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAggregate\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAgency 1\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAgency 2\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAgency 3\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eAgency 4\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAgency 5\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eAgency 6\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eOverall Dose Scores\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e2.3 (0.6)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e2.7 (0.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e2.7 (0.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e1.5 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e2.3 (0.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e2.3 (1.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e2.3 (0.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 1: Exploration\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u0026dagger;\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 2: Adoption\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e2.4 (0.6)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e2.7 (0.8)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e1.5 (1.6)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e2.4 (1.1)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e2.0 (1.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate fit\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.0 (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate feasibility\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e3.0 (0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate clinical utility and validity\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.5 (1.2)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate training requirements\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.3 (1.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate funding source\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.3 (1.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate benefits and costs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.4 (1.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e-\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eValidate adoption decision\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.0 (1.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e0\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 3: Preparation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e1.7 (0.6)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e2.2 (1.1)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e2.3 (1.0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e1.6 (1.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e0.8 (1.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e1.8 (1.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eGather and review treatment materials\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e0.6 (0.6)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEvaluate prospective adaptations\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.4 (1.3)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop adaptation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.0 (1.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003eN/A\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop training plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e2.6 (0.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDevelop implementation plan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e1.6 (1.1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePhase 4: Implementation\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e2.9 (0.1)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e2.9 (0.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e3.0 (0)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003cem\u003eNote.\u003c/em\u003e Dose scoring ranges from 0 (Nothing completed) to 3 (Mostly/All completed [5\u0026ndash;6 items]).\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e Agency implementation team made the decision to not adopt an EBP, therefore, did not progress past Phase 2.\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u0026dagger; Denotes that this is not applicable for the fidelity domain.\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e* Agency implementation team indicated that there were no training requirements while completing form.\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e- Indicates missing data.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFor participant responsiveness, ACT SMART facilitators rated agency implementation teams an overall average corresponding to \u0026ldquo;Very Engaged\u0026rdquo; (\u003cem\u003eM\u0026thinsp;=\u003c/em\u003e\u0026thinsp;4.0, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.50). The lowest average participant responsiveness rating was between \u0026ldquo;Slightly Engaged\u0026rdquo; and \u0026ldquo;Moderately Engaged\u0026rdquo; (\u003cem\u003eM\u0026thinsp;=\u003c/em\u003e\u0026thinsp;2.3, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.0) during the adoption decision phase of the toolkit. The highest average participant responsiveness rating was between \u0026ldquo;Very Engaged\u0026rdquo; to \u0026ldquo;Extremely Engaged\u0026rdquo; (\u003cem\u003eM\u003c/em\u003e\u0026thinsp;=\u0026thinsp;4.7, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.50) during the implementation phase (see Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). For facilitation meetings with multiple ACT SMART facilitators present, there was a 92.43% average agreement on participant responsiveness ratings.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab4\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eParticipant responsiveness to the ACT SMART Implementation Toolkit in aggregate and by individual agency implementation team\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAggregate\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAgency 1\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eAgency 2\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAgency 3\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003eAgency 4\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003eAgency 5\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cp\u003eAgency 6\u003c/p\u003e \u003cp\u003e\u003cem\u003eM\u003c/em\u003e (\u003cem\u003eSD\u003c/em\u003e)\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eOverall Participant Responsiveness Scores\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e4.0 (0.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e4.1 (0.7)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e4.6 (0.5)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e3.3 (0.1)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e\u003cb\u003e3.8 (0.3)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e\u003cb\u003e4.5 (0.4)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e\u003cb\u003e3.8 (0.9)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePhase 1: Exploration\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.8 (0.7)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePhase 2: Adoption\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2.3 (1.0)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e3.4\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e4.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3.5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePhase 3: Preparation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e3.5 (1.9)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e3.8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e4.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e4.0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e4.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e3.6\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ePhase 4: Implementation\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.7 (0.5)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cp\u003e4.3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003cem\u003eNote.\u003c/em\u003e Dose scoring ranges from 1 (Not at all engaged) to 5 (Extremely engaged).\u003c/td\u003e\u003c/tr\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003csup\u003e\u003cem\u003ea\u003c/em\u003e\u003c/sup\u003e Agency implementation team made the decision to not adopt an EBP, therefore, did not progress past Phase 2.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cb\u003eIndividual Agency Fidelity to the ACT SMART Toolkit.\u003c/b\u003e Across agencies, there was generally high adherence to toolkit activities; the lowest agency implementation team adhered to an overall average of 85.3% (\u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;20.2%) of required toolkit activities (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). While there was some variability in adherence across phases and activities by agency, there was no readily identifiable pattern of agencies consistently having lower or higher adherence compared to other agencies. Consistent with other results, the preparation phase appeared to have the lowest adherence ratings across agencies.\u003c/p\u003e \u003cp\u003eAgencies also all had generally high dose ratings for toolkit activities, except for the one agency (Agency 3) that chose not to adopt an EBP at the end of the Phase 2: Adoption (Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Like the ratings of adherence by agency, there was variability in dose ratings but no consistent identifiable patterns. Further, the preparation phase had the lowest dose ratings across agencies.\u003c/p\u003e \u003cp\u003eConsistent with both observations of adherence and dose ratings across agencies, all agencies also had relatively high ratings of participant responsiveness (Table\u0026nbsp;\u003cspan refid=\"Tab4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). The agency with the lowest average participant responsiveness rating was rated between \u0026ldquo;Moderately Engaged\u0026rdquo; to \u0026ldquo;Very Engaged\u0026rdquo; (\u003cem\u003eM\u0026thinsp;=\u003c/em\u003e\u0026thinsp;3.3, \u003cem\u003eSD\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0.1). However, in contrast to observations of adherence and dose ratings, agencies did not appear to have lower participant responsiveness during the preparation phase compared to other toolkit phases.\u003c/p\u003e \u003cp\u003e \u003cb\u003eDifferences in Fidelity to the ACT SMART Toolkit by Toolkit Phase.\u003c/b\u003e Our repeated measures ANOVAs to compare implementation strategy fidelity variables (i.e., adherence, dose, participant responsiveness) across phases revealed a significant main effect of toolkit phase for dose (\u003cem\u003eF\u003c/em\u003e(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e)\u0026thinsp;=\u0026thinsp;10.93, \u003cem\u003eMSE\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.190, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.005, η\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;.73, 95% CI [.15, .84]). However, there was not a significant main effect of toolkit phase for either adherence (\u003cem\u003eF\u003c/em\u003e(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e)\u0026thinsp;=\u0026thinsp;2.43, \u003cem\u003eMSE\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.035, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.116, η\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;.38, 95% CI [0, .57]) or participant responsiveness (\u003cem\u003eF\u003c/em\u003e(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e)\u0026thinsp;=\u0026thinsp;.2.31, \u003cem\u003eMSE\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.258, \u003cem\u003ep\u003c/em\u003e\u0026thinsp;=\u0026thinsp;.128, η\u003csup\u003e2\u003c/sup\u003e\u0026thinsp;=\u0026thinsp;.37, 95% CI [0, .69]).\u003c/p\u003e \u003cp\u003eUsing the Bonferroni post-hoc tests with local error terms to further examine the significant main effect of toolkit phase on dose, we found that the average dose rating during the preparation phase (Phase 3) of the toolkit was significantly lower than the average dose rating during the implementation phase (Phase 4) of the toolkit (\u003cem\u003ed\u003c/em\u003e\u0026thinsp;=\u0026thinsp;2.93, 95% CI [0.59, 5.20].\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003e\u003cb\u003eFidelity to the ACT SMART Toolkit.\u003c/b\u003e Our investigation used an instrumental case study approach to evaluate implementation strategy fidelity to the ACT SMART Toolkit by assessing observational descriptive ratings of adherence, dose, and participant responsiveness. Our evaluation provides one of the first models of assessing fidelity to a blended, multi-phased implementation strategy and important insights into both the potential for ASD community-based agencies to use the toolkit effectively and implementation strategy fidelity more broadly.\u003c/p\u003e \u003cp\u003eOverall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were relatively high, which supports the potential for the toolkit to be used with fidelity in ASD community agencies. Given that EBPs for ASD are often inconsistently or mis-implemented in community settings despite their potential to improve outcomes for a growing clinical population, understanding effective use of implementation strategies, such as the ACT SMART Toolkit, could contribute to reducing the EBP research-to-practice gap (\u003cspan additionalcitationids=\"CR21 CR22 CR23\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAlthough we found fidelity to be high overall, there was some variability in implementation strategy fidelity by toolkit phase. Specifically, we found that dose was significantly lower in the preparation phase (Phase 3) compared to the implementation phase (Phase 4). One possible rationale for this finding is that there were substantial differences in demands for toolkit activities by phase. Indeed, the preparation phase required gathering materials, evaluating prospective adaptations, and developing training and adaptation plans whereas the implementation phase required carrying out and evaluating the developed plans. Indeed, there were both lower adherence and dose ratings for toolkit activities such as developing adaptation and implementation plans compared to toolkit activities related to evaluating treatments, funding, and training. Thus, the lower dose in the preparation phase may reflect the need to lower the amount or intensity of toolkit activities required to better align with ASD community agency\u0026rsquo;s capacity to plan for implementation. Considering recently identified context-specific barriers and facilitators to the ACT SMART Toolkit would also likely be critical to enhancing Phase 4 (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAnother potential rationale for significantly lower dose during the preparation phase compared to the implementation phase may be that ASD community agencies perceived greater value in implementing the chosen EBP than in planning for its implementation. While agency implementation teams were rated as moderately to very engaged during the preparation phase, it is unclear how well facilitators were able to emphasize the important relationship between planning and implementation. However, researchers have recently proposed that fostering this understanding is necessary to support successful and sustainable implementation (\u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e). Thus, the ACT SMART Toolkit may also benefit from incorporating a greater focus on the practical importance of planning for implementation of EBPs.\u003c/p\u003e \u003cp\u003e\u003cb\u003eImplementation Strategy Fidelity Theory.\u003c/b\u003e Taken together, our instrumental case study assessment of fidelity to the ACT SMART Toolkit and exploration of the potential relationship between fidelity and EBP use within ASD community agencies notably provide one of the first models of assessing implementation strategy fidelity. Although a considerable amount of research has been conducted on \u003cem\u003eintervention fidelity\u003c/em\u003e, few researchers have explored \u003cem\u003eimplementation strategy fidelity\u003c/em\u003e (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e). For example, Slaughter et al. (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e) found that no studies reporting on fidelity to implementation included a specific definition or theoretical framework for assessing implementation strategy fidelity. To our knowledge, only Berry and colleagues (\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e) recently adapted the Conceptual Framework for Implementation Fidelity to guide their evaluation of fidelity to practice facilitation as a strategy to improve primary care practices\u0026rsquo; adoption of evidence-based guidelines for cardiovascular disease.\u003c/p\u003e \u003cp\u003eDespite limited research, evaluating and understanding implementation strategy fidelity has important implications and is identified as a research priority within dissemination and implementation science (\u003cspan additionalcitationids=\"CR45 CR46 CR47\" citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e). High fidelity to an implementation strategy may be reflective of other important implementation outcomes, such as high acceptability, appropriateness, and feasibility (\u003cspan citationid=\"CR57\" class=\"CitationRef\"\u003e57\u003c/span\u003e, \u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e). Further, implementation strategy fidelity may inform determination of which components of a strategy are required to produce change and which can be removed or adapted to account for varying contextual characteristics (\u003cspan additionalcitationids=\"CR42\" citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e). This knowledge may allow for demand optimization when the implementation strategy is being used, which may be particularly important when users of an implementation strategy have competing priorities or contextual factors that make completing the entirety of a blended implementation strategy difficult (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFrom our instrumental case study of ACT SMART Toolkit fidelity, we have demonstrated that fidelity to blended, multi-phased implementation strategies is possible. Further, we have highlighted that implementation strategy fidelity may vary according to differing components of a strategy, such as components focusing on preparation for implementation versus components focusing on implementation itself. We also observed that implementation strategy fidelity may vary by context. Here, implementation strategy fidelity was observed to vary across different ASD community agencies using the ACT SMART Toolkit. These findings suggest that a next step to further understand implementation strategy fidelity may be investigating shifts across both strategy content and context. Importantly, increasing this understanding could then also inform commonly needed adaptations to improve implementation strategy fidelity.\u003c/p\u003e \u003cp\u003e \u003cb\u003eStrengths.\u003c/b\u003e We propose a main strength of our investigation is that we demonstrate one of the first instrumental case studies to consider fidelity to a blended, multi-phased implementation strategy. Importantly, our assessment of fidelity to the ACT SMART Toolkit may be able to provide a framework for other evaluations of implementation strategy fidelity and inform the underlying theory of implementation strategy fidelity. Within our evaluation, we also importantly found overall high fidelity to the toolkit within ASD community-based agencies and identified potential ways in which to optimize demands of the toolkit and increase sustainability.\u003c/p\u003e \u003cp\u003e \u003cb\u003eLimitations.\u003c/b\u003e In contrast, important limitations of our investigation include potential issues with measurement of specific implementation strategy fidelity variables. For example, Berry and colleagues (\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e) recently considered participant responsiveness as a moderator of implementation strategy fidelity rather than a component of fidelity itself, as it was considered in our analysis. Moreover, the potential issues with measurement may have been compounded by the fact that standard measures were not used for dose and participant responsiveness. However, as an emerging field, implementation science often faces issues related to measurement and standardized measures specific to implementation strategy fidelity have not yet been developed (\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e, \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e, \u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e). Researchers have developed some standard measures for intervention fidelity, and these may be able to be adapted to assess implementation strategy fidelity in the future (\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eAnother potential limitation in our investigation is that there were different raters for adherence, dose, and participant responsiveness. While an independent observer rated adherence and dose for each implementation team, participant responsiveness was rated by a facilitator following implementation teams\u0026rsquo; facilitation meetings. Although this presents potential for bias, direct observation by independent observers and even implementers have been found to be more accurate than collecting reports directly from participants (\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e). Further, when two facilitators independently gave ratings for participant responsiveness, there were high rates of agreement.\u003c/p\u003e \u003cp\u003eMoreover, while we were generally able to assess implementation strategy fidelity by toolkit phase and activities, we were unable to assess all variables for all activities and by toolkit facet (i.e., web-based interface versus facilitation meetings). Thus, we are unable to make conclusions about all activities and the impact of the blended nature of the toolkit on implementation strategy fidelity. Further, our results may not generalize to discrete implementation strategies, which may benefit from their own instrumental case studies.\u003c/p\u003e \u003cp\u003eLastly, the most important limitation of our assessment of fidelity to the ACT SMART Toolkit was the limited sample size that rendered us under-powered to fully evaluate relationships between implementation strategy fidelity and EBP use. Moreover, our limited sample size also precluded us from considering additional factors such as implementation team and provider demographics and organizational climate within ASD community agencies. While we were able to observe variable implementation strategy fidelity across ASD community agencies, we were not yet able to identify consistent patterns related to higher or lower implementation strategy fidelity. However, there is evidence that some of these factors may moderate the relationship between implementation strategy fidelity to the ACT SMART toolkit and increased EBP use (\u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e61\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFuture research would benefit from consideration of potential moderators of implementation strategy fidelity and utilizing standard measures and independent raters (\u003cspan additionalcitationids=\"CR60 CR61 CR62 CR63\" citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e). In addition, future studies may benefit from a design intended to systematically evaluate fidelity to all components of a strategy. These lines of research may provide further insight into both effective use of the ACT SMART Toolkit as well as the advancing the field of implementation science more broadly.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eBy utilizing an instrumental case study approach, we advanced understanding of effective use of the ACT SMART Toolkit as well as the theory of implementation strategy fidelity more broadly. We found that the ACT SMART Toolkit has potential to be used with high fidelity in ASD community-based agencies. However, we also found that there was some variability in fidelity among toolkit phases, which points to possible adaptations needed to improve toolkit use even further. Considering adaptations may be critical as these findings may reflect that fidelity to blended, multi-phased implementation strategies is dynamic and affected by both strategy content and context. By increasing the use of and fidelity to effective implementation strategies that facilitate EBP adoption, utilization and sustainment within community-based settings, there is potential to increase overall public health.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eASD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAutism Spectrum Disorder\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEBP\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eEvidence-Based Practice\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eACT SMART Toolkit\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAutism Community Toolkit:Systems to Measure and Adopt Research-Based Treatments\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEPIS model\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eExploration, Preparation, Implementation, Sustainment model\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eABA\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eApplied Behavior Analysis\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cu\u003eEthics approval and consent participate\u003c/u\u003e: All participants provided informed consent.\u0026nbsp;The present study received exempt status from IRB review from (MASKED) to conduct secondary analysis. Data was collected while the senior author was at (MASKED), under the IRB approval # (MASKED).\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eConsent for publication\u003c/u\u003e: Not applicable\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAvailability of data and materials:\u003c/u\u003e The datasets used and/or analyzed during the current study are available from the senior author (MASKED) upon reasonable request.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eCompeting interests:\u003c/u\u003e The authors declare that they have no competing interests.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eFunding:\u003c/u\u003e This work was supported by (MASKED). Funding was used for the design and data collection of the pilot study of the ACT SMART toolkit.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAuthors\u0026rsquo; contributions:\u003c/u\u003e (First Author) contributed to conceptualization, selection of methodology, formal analysis, and writing of the original draft of the present manuscript. (Senior Author) conducted the pilot study of the ACT SMART Toolkit and contributed to conceptualization, selection of methodology, and review and editing of the present manuscript. Both authors read and approved the final manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAcknowledgments:\u003c/u\u003e (Senior Author) would like to express gratitude to the (MASKED), who provided valuable insight and recommendations as the Toolkit was developed. Additionally, (Senior Author) would like to thank Dr. Patricia Chamberlain for permitting the adaptation of the Stages of Implementation Completion. (First Author) and (Senior Author) would also like to acknowledge (MASKED), who provided invaluable statistical consultation.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eBlaxill M, Rogers T, Nevison C. Autism Tsunami: The Impact of Rising Prevalence on the Societal Cost of Autism in the United States. J Autism Dev Disord. 2022; 52(6):2627\u0026ndash;43. doi:10.1007/s10803-021-05120-7\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eLeigh JP, Du J. Brief Report: Forecasting the Economic Burden of Autism in 2015 and 2025 in the United States. J Autism Dev Disord. 2015; 45(12):4135\u0026ndash;9. doi:10.1007/s10803-015-2521-7\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eMaenner MJ, Shaw KA, Bakian AV, Bilder DA, Durkin MS, Esler A, et al. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years \u0026mdash; Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2018. MMWR Surveill Summ. 2021;70(11):1\u0026ndash;16. doi:10.15585/mmwr.ss7011a1\u003c/li\u003e\n \u003cli\u003eAmerican Psychiatric Association. Autism spectrum disorder. In: Diagnostic and statistical manual of mental disorders. 5th ed. Arlington, VA: American Psychiatric Association; 2013. doi:10.1176/appi.books.9780890425596\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eLai MC, Lombardo MV, Baron-Cohen S. Autism. The Lancet. 2014;383(9920):896\u0026ndash;910. doi:10.1016/S0140-6736(13)61539-1\u003c/li\u003e\n \u003cli\u003eSimonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric Disorders in Children With Autism Spectrum Disorders: Prevalence, Comorbidity, and Associated Factors in a Population-Derived Sample. Journal of the American Academy of Child \u0026amp; Adolescent Psychiatry. 2008;47(8):921\u0026ndash;9. doi:10.1097/CHI.0b013e318179964f\u003c/li\u003e\n \u003cli\u003eHattier MA, Matson JL, Belva BC, Horovitz M. The occurrence of challenging behaviours in children with autism spectrum disorders and atypical development. Developmental Neurorehabilitation. 2011;14(4):221\u0026ndash;9. doi: 10.3109/17518423.2011.573836\u003c/li\u003e\n \u003cli\u003eHorner RH, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: A research synthesis. Journal of Autism and Developmental Disorders. 2002;32(5):423\u0026ndash;46. doi:10.1023/A:1020593922901\u003c/li\u003e\n \u003cli\u003eStevens E, Atchison A, Stevens L, Hong E, Granpeesheh D, Dixon D, Linstead E. A cluster analysis of challenging behaviors in autism spectrum disorder. In 2017 16\u003csup\u003eth\u003c/sup\u003e IEEE International Conference on Machine Learning and Applications (ICMLA); 2017 December; IEEE. p.661-6.\u003c/li\u003e\n \u003cli\u003eKim SY, Bottema-Beutel K. A meta regression analysis of quality of life correlates in adults with ASD. Research in Autism Spectrum Disorders. 2019;63:23\u0026ndash;33. doi: 10.1016/j.rasd.2018.11.004\u003c/li\u003e\n \u003cli\u003eMason D, Mackintosh J, McConachie H, Rodgers J, Finch T, Parr JR. Quality of life for older autistic people: The impact of mental health difficulties. Research in Autism Spectrum Disorders. 2019;63:13\u0026ndash;22. doi:10.1016/j.rasd.2019.02.007\u003c/li\u003e\n \u003cli\u003eBottema-Beutel K, Kapp SK, Lester JN, Sasson NJ, Hand BN. Avoiding Ableist Language: Suggestions for Autism Researchers. Autism in Adulthood. 2021;3(1):18\u0026ndash;29. doi:10.1089/aut.2020.0014\u003c/li\u003e\n \u003cli\u003ePitney J. Lifetime social cost. Autism Politics and Policy [Internet]. Feb 6, 2020. Available from http://www.autismpolicyblog.com/2020/02/lifetime-social-cost.html\u003c/li\u003e\n \u003cli\u003eRobertson SM. Neurodiversity, Quality of Life, and Autistic Adults: Shifting Research and Professional Focuses onto Real-Life Challenges. DSQ. 2010;30(1). doi: 10.18061/dsq.v30i1.1069\u003c/li\u003e\n \u003cli\u003eKing M, Bearman P. Diagnostic change and the increased prevalence of autism. International Journal of Epidemiology. 2009;38(5):1224\u0026ndash;34. doi:10.1093/ije/dyp261\u003c/li\u003e\n \u003cli\u003eEapen V, Črnčec R, Walter A. Clinical outcomes of an early intervention program for preschool children with Autism Spectrum Disorder in a community group setting. BMC Pediatr. 2013;13(1):3. doi:10.1186/1471-2431-13-310.1186/1471-2431-13-3\u003c/li\u003e\n \u003cli\u003eHorlin C, Falkmer M, Parsons R, Albrecht MA, Falkmer T. The Cost of Autism Spectrum Disorders. Mulle JG, editor. PLoS ONE. 2014;9(9):e106552. doi:10.1371/journal.pone.0106552\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eVinen Z, Clark M, Paynter J, Dissanayake C. School Age Outcomes of Children with Autism Spectrum Disorder Who Received Community-Based Early Interventions. J Autism Dev Disord. 2018;48(5):1673\u0026ndash;83. doi:10.1007/s10803-017-3414-8\u003c/li\u003e\n \u003cli\u003eElder JH, Brasher S, Alexander B. Identifying the Barriers to Early Diagnosis and Treatment in Underserved Individuals with Autism Spectrum Disorders (ASD) and Their Families: A Qualitative Study. Issues in Mental Health Nursing. 2016;37(6):412\u0026ndash;20. doi: 10.3109/01612840.2016.1153174\u003c/li\u003e\n \u003cli\u003eDrahota A, Meza RD, Bustos TE, Sridhar A, Martinez JI, Brikho B, et al. Implementation-as-Usual in Community-Based Organizations Providing Specialized Services to Individuals with Autism Spectrum Disorder: A Mixed Methods Study. Adm Policy Ment Health. 2021;48(3):482\u0026ndash;98. doi:10.1007/s10488-020-01084-5\u003c/li\u003e\n \u003cli\u003ePaynter JM, Ferguson S, Fordyce K, Joosten A, Paku S, Stephens M, et al. Utilisation of evidence-based practices by ASD early intervention service providers. Autism. 2017;21(2):167\u0026ndash;80. doi:10.1177/1362361316633032\u003c/li\u003e\n \u003cli\u003ePickard K, Meza R, Drahota A, Brikho B. They\u0026rsquo;re Doing What? A Brief Paper on Service Use and Attitudes in ASD Community-Based Agencies. Journal of Mental Health Research in Intellectual Disabilities. 2018;11(2):111\u0026ndash;23. doi:10.1080/19315864.2017.1408725\u003c/li\u003e\n \u003cli\u003eWong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, et al. Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder: A Comprehensive Review. J Autism Dev Disord. 2015;45(7):1951\u0026ndash;66. doi:10.1007/s10803-014-2351-z\u003c/li\u003e\n \u003cli\u003eWood JJ, McLeod BD, Klebanoff S, Brookman-Frazee L. Toward the Implementation of Evidence-Based Interventions for Youth With Autism Spectrum Disorders in Schools and Community Agencies. Behavior Therapy. 2015;46(1):83\u0026ndash;95. doi:10.1016/j.beth.2014.07.003\u003c/li\u003e\n \u003cli\u003eDrahota A, Chlebowski C, Stadnick N, Baker-Ericz\u0026eacute;n MJ, Brookman-Frazee L. The dissemination and implementation of behavioral treatments for anxiety in ASD. In: Kerns C, Renno P, Storch A, Kendall PC, Wood JJ, editors. Anxiety in children and adolescents with autism spectrum disorder: Evidence-based assessment and treatment. Atlanta, GA: Elsevier; 2017. p.231-249.\u003c/li\u003e\n \u003cli\u003eDrahota A, Meza R, Martinez JI. The autism-community toolkit: Systems to measure and adopt research-based treatments. 2014. Available from www.actsmartoolkit.com\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eGomez E, Drahota A, Stahmer AC. Choosing strategies that work from the start: A mixed methods study to understand effective development of community\u0026ndash;academic partnerships. Action Research. 2021;19(2):277\u0026ndash;300. doi:10.1177/1476750318775796\u003c/li\u003e\n \u003cli\u003eDrahota A, Aarons GA, Stahmer AC. Developing the Autism Model of Implementation for Autism spectrum disorder community providers: study protocol. Implementation Sci. 2012;7(1):85. doi:10.1186/1748-5908-7-85\u003c/li\u003e\n \u003cli\u003eAarons GA, Hurlburt M, Horwitz SM. Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Adm Policy Ment Health. 2011;38(1):4\u0026ndash;23. doi:10.1007/s10488-010-0327-7\u003c/li\u003e\n \u003cli\u003eDrahota A., Meza R., Martinez JI, Sridhar A., Bustos TE, Tschida J., Stahmer A, Aarons, GA. Feasibility, Acceptability, and Utility of the ACT SMART Implementation Toolkit. In preparation.\u003c/li\u003e\n \u003cli\u003eSridhar A, Drahota A. Preliminary effectiveness of the ACT SMART implementation toolkit: facilitating evidence-based practice implementation in community-based autism organizations. International Journal of Developmental Disabilities. 2022:1-6. doi:10.1080/20473869.2022.2065448\u003c/li\u003e\n \u003cli\u003eSridhar A, Drahota A, Walsworth K. Facilitators and barriers to the utilization of the ACT SMART Implementation Toolkit in community-based organizations: a qualitative study. Implement Sci Commun. 2021;2(1):55. doi:10.1186/s43058-021-00158-1\u003c/li\u003e\n \u003cli\u003eAllen JD, Shelton RC, Emmons KM, Linnan LA (2018). Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Brownson RC, Colditz GA, Proctor, EK, editors. Dissemination and implementation research in health: Translating science to practice. Oxford Scholarship Online; 2017. https://doi.org/10.1093/oso/9780190683214.003.0016\u003c/li\u003e\n \u003cli\u003eMowbray CT, Holter MC, Teague GB, Bybee D. Fidelity Criteria: Development, Measurement, and Validation. American Journal of Evaluation. 2003;24(3):315\u0026ndash;40. doi:10.1177/109821400302400303\u003c/li\u003e\n \u003cli\u003eSlaughter SE, Hill JN, Snelgrove-Clarke E. What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review. Implementation Sci. 2015;10(1):129. doi:10.1186/s13012-015-0320-3\u003c/li\u003e\n \u003cli\u003eDusenbury L. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research. 2003;18(2):237\u0026ndash;56. doi:10.1093/her/18.2.237\u003c/li\u003e\n \u003cli\u003eTeague GB. Fidelity. Implementation Research Institute Presentation; 2013.\u003c/li\u003e\n \u003cli\u003eCentury J, Rudnick M, Freeman C. A Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge. American Journal of Evaluation. 2010;31(2):199\u0026ndash;218. doi:10.1177/1098214010366173\u003c/li\u003e\n \u003cli\u003eMetz A, Bartley L. Implementation teams: A stakeholder view of leading and sustaining change. In: Implementation Science 3.0. Springer, Cham; 2020. p.199-225.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eCross W, West J. Examining implementer fidelity: conceptualising and measuring adherence and competence. Journal of Children\u0026rsquo;s Services. 2011;6(1):18\u0026ndash;33. doi:10.5042/jcs.2011.0123\u003c/li\u003e\n \u003cli\u003eKirk MA, Haines ER, Rokoske FS, Powell BJ, Weinberger M, Hanson LC, et al. A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Translational Behavioral Medicine. 2021;11(1):21\u0026ndash;33. doi: 10.1093/tbm/ibz178\u003c/li\u003e\n \u003cli\u003eMihalic S. The importance of implementation fidelity.\u0026nbsp;Emotional and Behavioral Disorders in Youth\u003cem\u003e.\u0026nbsp;\u003c/em\u003e2004;4(4):83-105. http://www.incredibleyears.com/wp-content/uploads/fidelity-importance.pdf\u003cu\u003e\u0026nbsp;\u003c/u\u003e\u003c/li\u003e\n \u003cli\u003ePerez Jolles M, Lengnick-Hall R, Mittman BS. Core Functions and Forms of Complex Health Interventions: a Patient-Centered Medical Home Illustration. J GEN INTERN MED. 2019;34(6):1032\u0026ndash;8. doi:10.1007/s11606-018-4818-7\u003c/li\u003e\n \u003cli\u003eThe CIPHER team, Haynes A, Brennan S, Redman S, Williamson A, Gallego G, et al. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implementation Sci. 2015;11(1):23. doi:10.1186/s13012-016-0378-6\u003c/li\u003e\n \u003cli\u003eMoore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258\u0026ndash;h1258. doi:10.1136/bmj.h1258\u003c/li\u003e\n \u003cli\u003eAkiba CF, Powell BJ, Pence BW, Nguyen MXB, Golin C, Go V. The case for prioritizing implementation strategy fidelity measurement: benefits and challenges. Translational Behavioral Medicine. 2022;12(2):335\u0026ndash;42. doi:10.1093/tbm/ibab138\u003c/li\u003e\n \u003cli\u003eAkiba CF, Powell BJ, Pence BW, Muessig K, Golin CE, Go V. \u0026ldquo;We start where we are\u0026rdquo;: A qualitative study of barriers and pragmatic solutions to the assessment and reporting of implementation strategy fidelity [Internet]. In Review; 2022 [cited 2022 Jul 13]. Available from: https://www.researchsquare.com/article/rs-1626073/v1\u003c/li\u003e\n \u003cli\u003ePowell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health. 2019;7:3. doi:10.3389/fpubh.2019.000031.\u003c/li\u003e\n \u003cli\u003eBerry CA, Nguyen AM, Cuthel AM, Cleland CM, Siman N, Pham-Singer H, et al. Measuring Implementation Strategy Fidelity in HealthyHearts NYC: A Complex Intervention Using Practice Facilitation in Primary Care. American Journal of Medical Quality. 2021;36(4):270\u0026ndash;6. doi:10.1177/1062860620959450.\u003c/li\u003e\n \u003cli\u003eKourouche S, Curtis K, Munroe B, Watts M, Balzer S, Buckley T. Implementation strategy fidelity evaluation for a multidisciplinary Chest Injury Protocol (ChIP). Implement Sci Commun. 2021;2(1):86. doi:10.1186/s43058-021-00189-8\u003c/li\u003e\n \u003cli\u003eCrowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100. doi:10.1186/1471-2288-11-100\u003c/li\u003e\n \u003cli\u003eChamberlain P, Brown CH, Saldana L. Observational measure of implementation progress in community based settings: The Stages of implementation completion (SIC). Implementation Sci. 2011;6(1):116. doi:10.1186/1748-5908-6-116\u003c/li\u003e\n \u003cli\u003eDrahota A, Martinez JI. ACT SMART Milestones and Activity Fidelity Forms. Unpublished measure. 2014.\u003c/li\u003e\n \u003cli\u003ePinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. BMJ. 2017;356:i6795.\u003c/li\u003e\n \u003cli\u003ePowell BJ, Haley AD, Patel SV, Amaya-Jackson L, Glienke B, Blythe M, et al. Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS). Implement Sci Commun. 2020;1(1):9. doi:10.1186/s43058-020-00009-5\u003c/li\u003e\n \u003cli\u003eLeal Filho W, Skanavis C, Kounani A, Brandli LL, Shiel C, Pa\u0026ccedil;o A do, et al. The role of planning in implementing sustainable development in a higher education context. Journal of Cleaner Production. 2019;235:678\u0026ndash;87. doi:10.1016/j.jclepro.2019.06.322\u003c/li\u003e\n \u003cli\u003eProctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38(2):65\u0026ndash;76. doi:10.1007/s10488-010-0319-7\u003c/li\u003e\n \u003cli\u003eWeiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implementation Sci. 2017;12(1):108. doi:10.1186/s13012-017-0635-3\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eLewis CC, Dorsey C. Advancing implementation science measurement.\u0026nbsp;In: Albers B, Shlonsky A, Mildon R, editors. Implementation Science 3.0. Springer Nature; 2020. p.227.\u003c/li\u003e\n \u003cli\u003eIbrahim S, Sidani S. Fidelity of Intervention Implementation: A Review of Instruments. Health. 2015;07(12):1687\u0026ndash;95. doi:10.4236/health.2015.712183\u003c/li\u003e\n \u003cli\u003eHasson H, Blomberg S, Dun\u0026eacute;r A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Sci. 2012;7(1):23. doi:10.1186/1748-5908-7-23\u003c/li\u003e\n \u003cli\u003eBarber JP, Gallop R, Crits-Christoph P, Frank A, Thase ME, Weiss RD, et al. The role of therapist adherence, therapist competence, and alliance in predicting outcome of individual drug counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229\u0026ndash;40. doi:10.1080/10503300500288951\u003c/li\u003e\n \u003cli\u003eHogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology. 2008;76(4):544\u0026ndash;55. doi:10.1037/0022-006X.76.4.544\u003c/li\u003e\n \u003cli\u003eMcHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, et al. Fidelity Outcomes in the National Implementing Evidence-Based Practices Project. PS. 2007;58(10):1279\u0026ndash;84. doi:10.1176/ps.2007.58.10.1279\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"implementation-science-communications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"iscm","sideBox":"Learn more about [Implementation Science Communications](https://implementationsciencecomms.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ISCM/default.aspx","title":"Implementation Science Communications","twitterHandle":"@ImplementSci","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"fidelity, autism spectrum disorder, blended implementation strategy, case study, implementation fidelity ","lastPublishedDoi":"10.21203/rs.3.rs-1930423/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1930423/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Although evidence-based practices (EBPs) have been shown to improve a variety of outcomes for autistic children, they are often inconsistently implemented or not implemented in community settings where many autistic children receive usual care services. One blended multi-phased implementation strategy developed to support the adoption and implementation of EBPs for autism spectrum disorder (ASD) in community-based settings is the Autism Community Toolkit: Systems to Measure and Adopt Research-Based Treatments (ACT SMART Toolkit). Guided by an adapted version of the EPIS, the ACT SMART Toolkit is comprised multiple phases, each with steps and activities to be completed by agency implementation teams. In the present study, we evaluated implementation strategy fidelity, given the potential for important insights into the use of the toolkit and limited information on the phenomenon of implementation strategy fidelity more broadly.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e We used an instrumental case study approach to assess fidelity to the ACT SMART Toolkit during its pilot study with six ASD community agency implementation teams. We assessed adherence, dose, and implementation team responsiveness for each phase and activity of the toolkit at both an aggregate and individual agency level. We also conducted repeated measures ANOVAs to determine whether implementation strategy fidelity significantly differed by toolkit phase. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e Overall, we found that adherence, dose, and participant responsiveness to the ACT SMART Toolkit were high, with some variability by toolkit phase and activity, as well as by ASD community agency. There was a significant main effect of toolkit phase for dose (\u003cem\u003eF\u003c/em\u003e(2,8) = 10.93, \u003cem\u003eMSE \u003c/em\u003e= .190, \u003cem\u003ep\u003c/em\u003e = .005, η\u003csup\u003e2\u003c/sup\u003e = .73), such that dose was significantly lower during the preparation phase as compared to the implementation phase of the toolkit. \u003c/p\u003e\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e Our instrumental case study evaluation of fidelity to the ACT SMART Toolkit demonstrated potential for the strategy to be used with fidelity in ASD community-based agencies. Findings related to variability of implementation strategy fidelity in the present study may also inform future adaptations to the toolkit and point to broader trends of how implementation strategy fidelity may vary by content and context.\u0026nbsp;\u003c/p\u003e","manuscriptTitle":"Fidelity to the ACT SMART Toolkit: An Instrumental Case Study of Implementation Strategy Fidelity","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-08-08 17:59:56","doi":"10.21203/rs.3.rs-1930423/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"","date":"2022-09-25T04:57:59+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-09-25T02:55:35+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-08-05T07:13:18+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2022-08-04T23:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2022-08-04T23:00:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"Implementation Science Communications","date":"2022-08-04T12:50:49+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"implementation-science-communications","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"iscm","sideBox":"Learn more about [Implementation Science Communications](https://implementationsciencecomms.biomedcentral.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/ISCM/default.aspx","title":"Implementation Science Communications","twitterHandle":"@ImplementSci","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cfe7c113-ed3e-4f21-a09c-b44366afa1cb","owner":[],"postedDate":"August 8th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2023-10-16T21:05:15+00:00","versionOfRecord":{"articleIdentity":"rs-1930423","link":"https://doi.org/10.1186/s43058-023-00434-2","journal":{"identity":"implementation-science-communications","isVorOnly":false,"title":"Implementation Science Communications"},"publishedOn":"2023-05-16 20:51:11","publishedOnDateReadable":"May 16th, 2023"},"versionCreatedAt":"2022-08-08 17:59:56","video":"","vorDoi":"10.1186/s43058-023-00434-2","vorDoiUrl":"https://doi.org/10.1186/s43058-023-00434-2","workflowStages":[]},"version":"v1","identity":"rs-1930423","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1930423","identity":"rs-1930423","version":["v1"]},"buildId":"-HB7Z8yhvgn0wM9Nzuekk","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}