Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention

Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention Yu Chen Lin, Maddie Offstein, Cassidy Malner, Angel Williams, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4707882/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background. Detailed time and cost data are often lacking in implementation science, particularly in school-based interventions. In a pilot intervention in one Chicago Public Schools’ geographic network, a Network Specialist was hired to provide schools with tailored technical assistance (TA) to support compliance with over 50 health-related policies (the Healthy CPS initiative). This study describes the methods for developing and implementing an activity-based time tracking tool to assess the Network Specialist’s fidelity, time, and cost in providing TA using a Multi-Tiered System of Supports framework (“Tier 1” universal support, “Tier 2” targeted support, and “Tier 3” intensive, individualized support). Methods. The tool was developed in close collaboration with the Network Specialist to capture the Specialist’s interactions with schools between 2020–2023. Key informant interviews and asynchronous post-hoc feedback were qualitatively analyzed to assess the Specialist’s feedback on the tool. Descriptive statistics on school interactions, tiers of support provided, and domains of implementation support provided using the SISTER implementation science framework were generated from the tracking tool data. Differences in mean baseline Healthy CPS policy compliance based on the extent of schools’ interactions with the Specialist in each tier of support were computed using Wald tests. Results. The Specialist described the tracking tool as feasible and useful in providing tailored support and advocated for its continued use as the intervention is expanded to additional networks. The Specialist spent the highest proportion of time and costs (41.4%, $ 39,117) providing intensive Tier 3 supports, and those supports were targeted toward schools with the most need. Schools receiving the most Tier 3 supports had lower baseline Healthy CPS compliance of 64.4%, versus 78.4% and 67.3% for schools receiving low and medium levels of Tier 3 supports, respectively (p-value = 0.045). Conclusions. Expanded use of time and cost tracking is needed in implementation science, particularly for school-based interventions. Time tracking tools help collect data on intervention activities that inform decision-makers about how to implement interventions with fidelity. Our findings point to the value of using a collaborative, partner-engaged approach to developing the tracking tool with the end user to maximize its feasibility, usefulness, utilization, and sustainability. Activity-based time tracking school health and wellness policy implementation science costs community-engaged research Figures Figure 1 Contributions to the literature Few interventions assess the time and costs associated with implementation strategies used and activities performed, even more so in evaluating school-based interventions. The activity-based time tracking tool used in this study to support school-level health and wellness policy implementation revealed that the intervention targeted the most intensive supports to schools with the greatest need. Time tracking tools should be developed in close collaboration with the end user to enhance their feasibility and usefulness. BACKGROUND In the United States (U.S.), schools are required to comply with a multitude of health and wellness-related policies at the federal, state, and local (school district) levels ( 1 , 2 ). Keeping track of all of the policies, much less implementing them, is complex and there is no one-size-fits-all approach. In fact, Cook et al.’s School Implementation Strategies, Translating ERIC Resources (SISTER) framework identified 79 strategies that could be used to support implementation of school-based interventions grouped across nine domains: (a) use evaluative and iterative strategies, (b) provide interactive assistance, (c) adapt and tailor to context, (d) develop stakeholder interrelationships, (e) train and educate stakeholders, (f) support educators, (g) engage consumers, (h) use financial strategies, and (i) change infrastructure ( 3 ). Chicago Public Schools (CPS), one of the largest school districts in the U.S., struggled to support schools with implementation of over 50 federal, state, and local health and wellness-related policies. Recognizing this, in 2016, CPS created the Healthy CPS initiative to help schools navigate and implement these policies ( 4 ). For the first four years of the Healthy CPS initiative, the district’s Office of Student Health and Wellness (OSHW) tried to support schools through standardized technical assistance (TA). However, it became clear that with over 600 schools ( 5 ), centralized TA was not realistic and did not provide schools with the level of support they required. Instead, CPS’ OSHW wanted to pilot test providing more localized support to schools through their 17 geographic networks; each network is comparable to a small-medium sized school district elsewhere in the country, with an average of more than 14,000 students per district and ~ 15–35 schools per network ( 6 , 7 ). Each network is led by a Network Chief and each network provides administrative support, strategic direction, and leadership development to schools within the network ( 6 ). When CPS schools need specific supports beyond the network, they are typically referred to offices and units throughout the district (e.g., Children and Family Benefits, OSHW, etc.). However, in 2020, in a collaborative community-academic partnership with the Policy, Practice and Prevention Research Center at the University of Illinois Chicago, CPS piloted an intervention to support one network in the west side of Chicago with implementing Healthy CPS. To implement this intervention, CPS’ OSHW created a new Healthy CPS Network Specialist position to help schools implement these policies by providing TA and connecting schools to various resources ( 8 ). The Network Specialist used a Multi-Tiered System of Supports (MTSS) framework to provide schools with varying tiers of support in implementing Healthy CPS policies. Similar to the continuum of prevention model used to prevent substance use and to improve child welfare ( 9 , 10 ), the MTSS framework is used by the education sector to provide “Tier 1” universal support to all schools/students, “Tier 2” targeted support to a subset of schools/students that require additional assistance, and “Tier 3” individualized support to schools/students with ongoing and intensive needs ( 11 , 12 ). School-based interventions delivered using a MTSS framework have been shown to improve student mental health, social behaviors, school engagement, academic performance, and more ( 13 , 14 ). The Network Specialist’s activities were aligned with Cook et al.’s SISTER strategies ( 3 ). One of the goals with the Network Specialist intervention was to not only test the efficacy of the Network Specialist position in helping schools improve Healthy CPS compliance but also, equally important for the district leadership, to provide data for CPS to determine the nature of the supports provided by the Specialist, the time spent on providing supports by MTSS Tier, and the cost to do so. However, detailed time and cost data on intervention activities are often lacking in implementation science, even more so in evaluating school-based interventions ( 15 , 16 ). Various methods have been developed to fill this gap. A common costing method is the top-down approach, which is one that allocates the overall time and cost spent at the organization level to the underlying implementation activities ( 17 ). This type of allocation method relies heavily on the richness and details of data on intervention activities to approximate or assume the distribution of time and cost ( 17 ). This approach often suffers from lack of accuracy, especially for evaluating interventions. Another costing method is the bottom-up approach, which aggregates individual-level data to derive the overall costs ( 17 ). This approach is often used in medical billing and fee-for-service applications, and is very time consuming and labor intensive ( 17 ). More modern and mixed approaches have been developed to overcome the challenges in these traditional methods. Saldana et al. developed a costing approach that mapped intervention costs to different stages of implementation ( 18 ). Based on a business accounting method ( 19 ), Cidav et al. also detailed a time-driven activity-based costing approach that mapped costs to various implementation strategies ( 20 ). By using these approaches, researchers can directly compare the time and costs of various intervention activities performed and the implementation strategies utilized. Previous studies have used activity-based time tracking (ABTT) tools in primary care and substance use interventions to assess the cost of intervention activities ( 21 , 22 ). These studies highlight the utility of these approaches in implementation science, and the use of time tracking tools helps collect data on intervention activities that inform decision-makers about how to implement policies and interventions with fidelity. In the methods section that follows, we describe the development of an ABTT tool for use by the Network Specialist in supporting schools with policy implementation. Then, in the results, we ( 1 ) document the utility and feasibility of using the tool; ( 2 ) illustrate how the ABTT tool was used to assess the time and cost spent on providing supports by mode of support, MTSS Tier, and SISTER domain; and ( 3 ) show how the data from the ABTT tool were used to demonstrate that the Network Specialist provided support for schools with the greatest need. We hypothesized that the Network Specialist would spend a greater proportion of time providing intensive Tier 3 supports, and that those supports would predominantly be targeted toward schools with the most need as measured by baseline Healthy CPS policy compliance. To the best of our knowledge, an ABTT tool has not been used before in school-based interventions to support schools in implementing health and wellness policies. METHODS Development of the Tool Use of a Community-Engaged Process for Developing and Improving the Tool A community-engaged process ( 23 ) was taken to develop the ABTT tool, with iterative feedback from the Network Specialist and OSHW leadership to continuously improve the tracking tool for the end user (i.e., the Network Specialist) ( 24 ). Guided by existing literature on tracking implementation activities ( 18 , 20 ), the initial version of the tracking tool used in school years (SY) 2020–2022 was developed in Google Sheets, CPS’ preferred platform, to capture the Network Specialist’s day-to-day activities because it could be easily integrated into CPS’ existing TA tracking infrastructure housed in Google Sheets. The Network Specialist recorded each interaction with schools on a weekly basis (as close to real time as possible), including basic information about the nature of the interaction (e.g., date, names and roles of persons interacted with, mode of contact, duration). An open-ended text field was also provided for the Network Specialist to describe the topic and content of the interaction. However, upon reviewing the tracking tool data in 2022, it became apparent that it was not accurately capturing the details on the specific supports/strategies being used by the Specialist, nor could it be used to identify the MTSS Tiers of support provided by the Specialist. Thus, through a series of four meetings between the UIC research team and the Network Specialist, we developed a revised tool in Google Forms (for use in SY 2022–2023), similar to completing an electronic survey. The revised version of the tool was expanded to collect additional information on how TA support was delivered, including additional open-ended fields, close-ended questions on the most commonly provided support, and Healthy CPS topics discussed. This was done to standardize the tracking tool and to enhance its reliability. Standardization of tracking TA support across multiple end users was essential to the validity and sustainability of the tracking tool as CPS sought to expand its use to other CPS networks with the planned onboarding of additional Network Specialists. The use of the Google Form allowed for more streamlined data collection and reduced missing or inaccurate data through the use of forced responses and data validation. Figure 1 summarizes the main components of the tool presented in this study; Additional File 1 contains the specific questions and response options included in the tool for the items captured for this study. (A full-length version of the tool which captured specific Healthy CPS policy issues being addressed at each school is available upon request from the corresponding author.) Insert Fig. 1 about here Entries from both iterations of the ABTT tool were processed monthly by the UIC research team using Stata v.16 (StataCorp, College Station, TX, USA) to ensure data fidelity. Any inconsistencies or missing entries were reconciled with the Network Specialist via follow-up discussions and email communications. Summary statistics were also generated monthly and reviewed to monitor implementation fidelity and provide continuous quality improvement feedback to the Network Specialist on how to effectively use the tool. Between SY 2020–2023, a total of 78 interactions (total duration: 57.0 hours) took place between the Network Specialist and the research team for the purposes of continuous quality improvement of the tracking tool (e.g., via meetings/emails to clarify what interactions were being had). Face validity of the fields in the tracking tool was assessed through discussions with the Network Specialist to evaluate that their understanding of what each field was asking for aligned with what the research team had intended. Determining the SISTER Domains and Tiers of Support used by the Network Specialist Open-ended text fields, specifically questions 10 and 13 in the ABTT tool (see Additional File 1), were used by the Network Specialist to describe the topic and content of the interaction, resources shared, referrals made, and activities/strategies used to support schools’ health and wellness policy implementation efforts, as well as the tier of MTSS support provided. This open-ended format was used as Walsh-Bailey et al. found that an open-ended tool to record implementation strategies was more acceptable and feasible compared to a more structured tool forcing respondents to select from pre-populated options ( 25 ). Researchers also wanted the Network Specialist to be able to provide as much detail as possible, so that the interactions could be later reviewed and coded by the research team. To code the open-ended text field data, entries were exported from the ABTT tool into an Excel spreadsheet, and columns were added to code the entries using SISTER implementation strategy domains and definitions put forth by Cook et al. ( 3 ). Using these initial code definitions for the implementation strategy domains, the open-ended field data were independently coded by three members of the research team (two lead investigators and one research associate). After this initial review of the entries and application of the SISTER domain codes, the research team held multiple meetings to further refine the SISTER domain code definitions to be more specific to the work of the Network Specialist and identified specific SISTER implementation strategies within each SISTER domain that the Network Specialist was using to help schools achieve Healthy CPS compliance (e.g., prepare and identify champions was identified as a strategy used by the Network Specialist and defined as preparing and identifying Wellness Champions to become leaders in implementing Healthy CPS in their schools). The research team then re-reviewed the initial application of SISTER domain codes and resolved any differences through discussion (and in consultation with the Network Specialist when additional details were needed about the nature of the logged interaction). After confirming agreement on the application of the SISTER domains, the research team then coded for the specific SISTER implementation strategies within each domain that mapped to the regular activities of the Network Specialist, and similarly met to discuss how the codes were being applied and rectify any disagreements in application. Investigators arrived at a final codebook that contained 6 SISTER domains and 15 SISTER strategies. For the sake of brevity and due to the limited number of total interactions coded, data in this study is presented at the implementation strategy domain-level (and not at the more specific implementation strategy-level). In addition to applying SISTER domain and strategy codes, the ABTT tool data were analyzed by designating the MTSS Tier provided by the Network Specialist. The MTSS categories included Tiers 1, 2, and 3—with Tier 1 supports being the least involved supports that are provided to all schools and Tiers 2 and 3 representing more focused and the most targeted supports to schools, respectively, which were provided based on individual schools’ needs and Healthy CPS attainment status. When completing the log, the Network Specialist designated which tier of support they thought the interaction fell under – which the research team then reviewed when processing the data to ensure that the interaction was categorized correctly. The Network Specialist’s understanding of the differences in tiers was also periodically checked during process evaluation interviews throughout the study period. From the tool, 556 interactions were reviewed and only one interaction logged by the Network Specialist could not be categorized into any of the three tiers of support. Determining the Utility and Feasibility of the Tool Qualitative data were used to assess the utility and feasibility of using the ABTT tool. First, three key informant interviews between the Network Specialist and the UIC research team where the tool was discussed were held between 2021 and 2023. (Separate meetings were held with the Specialist to discuss tool development and refinement to ensure tool validity through continuous quality improvement of the data entry, but those meetings did not capture data on the use or feasibility of the tool.) Additionally, an asynchronous structured interview over email took place in May 2024 with the Network Specialist (one of the co-authors, CM) to gain insight into their perspective on the feasibility and usefulness of the tool ( 26 ). Data Sources for Examining the Implementation Costs and Whether the Specialist Supported Schools with the Most Need Cost Data Consistent with previous studies evaluating school-based interventions, costs were measured from an education sector perspective and included only the programmatic costs ( 27 , 28 ). This perspective is most relevant to decision-makers interested in implementing the intervention. Costs were grouped into three categories: personnel, travel, and equipment. Personnel costs were prorated based on the proportion of time the Network Specialist spent interacting with schools (13.7% in SY 2020–2021, 32.0% in SY 2021–2022, and 51.5% in SY 2022–2023), excluding their interactions with CPS, OSHW, and the research team that pertained to the design and planning of the intervention or the tracking tool. Travel and equipment costs were not prorated. Interactions between the Network Specialist and schools were relatively low in the first year due to the COVID-19 pandemic when Chicago schools were operating remotely and the Network Specialist was restricted in their capacity to interact with schools by the school district leadership. Personnel costs included the salary and benefits of the Network Specialist (1 full-time equivalent [FTE]) and the Healthy CPS Program Manager, who reported spending 10% of their time supporting the Network Specialist in implementing the intervention (0.1 FTE). Travel costs were derived from reimbursement forms submitted by the Network Specialist to CPS to cover their visits to schools to conduct on-site meetings with school leadership and provide TA. Mileage rates for travel were determined based on the standard mileage rate published by the Internal Revenue Service for business use ( 29 ). Equipment costs were self-reported and included the purchasing of CPS-issued electronic devices. Healthy CPS Scores To assess the extent to which the Network Specialist was providing targeted supports to schools with the greatest need at baseline, publicly available Healthy CPS annual data from SY 2018–2019 were compiled for each school through CPS’ School Profile Search ( 30 ). Healthy CPS for that SY assessed 32 policies pertaining to four health and wellness areas: 1) chronic disease management (e.g., whether staff completed the chronic conditions training), 2) health instruction (e.g., whether schools provided the required physical, nutrition, and sexual health education curriculum), 3) school health leadership and environments (e.g., whether the school has a Wellness Champion and team to facilitate wellness policy implementation, whether the school offered daily recess), and 4) health services (e.g., whether the school participated in dental and vision exam programs) ( 31 ). The Healthy CPS were collected through district administrative data and schools’ self-reported surveys. The SY 2018–2019 Healthy CPS score, computed as the averaged percentage score in complying with policies in each of the four health and wellness areas, serves as a baseline measure of a school’s need for TA before the Network Specialist intervention. Analysis Qualitative Analysis To determine feasibility and utility of the tool, the qualitative interview data and the asynchronous email data were analyzed by two study authors (MO, EJR) using MaxQDA Pro (VERBI GmbH, Berlin, Germany). The transcripts and email were analyzed to identify specific feedback from the Network Specialist about the tool and how it was being used. The qualitative data were organized according to broad categories of feasibility and utility. Most of the interview data focused on feasibility, particularly strengths and weaknesses of using the tool, whereas the final email exchange captured specific information on the utility or value of using the tool. Quantitative Analyses ABTT tool data were pulled from Google Sheets or Google Forms monthly between June 2020 and June 2023. Any missing or inconsistent data were resolved through discussions with the Network Specialist. If an activity pertained to multiple schools, the duration was viewed as the overall time it took to complete the activity across the schools; thus, the duration was recoded by splitting the time equally among the schools involved. For example, if the Network Specialist spent 10 minutes to draft and send an email blast to all intervention schools, the overall duration of that interaction would be divided and attributed equally to each of the schools. Cost by mode of interaction, tier of support, and SISTER implementation domains were computed by the percentage of time spent by the Network Specialist. For each tier of support, the schools served by the Specialist were categorized into tertiles (low, medium, and high) based on the total duration of interactions with the Network Specialist falling into the given tier. Mean SY 2018–2019 Healthy CPS scores were computed within each tier of support and tertile of interaction duration. A Wald test was performed to determine the statistical significance of differences in mean scores within each tier of support. All analyses were performed in Stata v.17 (StataCorp, College Station, TX, USA). RESULTS Between SY 2020–2021 and SY 2022–2023, the Network Specialist supported 33 unique schools with an average of 7,942 students per SY (Table 1 ). The students in the intervention (Network Specialist) network were predominantly Black and Hispanic, most students were eligible for free or reduced-price lunch, and about one-sixth of the students had limited English proficiency and required special education ( 32 ). This study was approved by the University of Illinois Chicago Institutional Review Board (protocol #2019 − 1161). Table 1 School and student characteristics of the intervention network, school years 2020–2023. Count or Mean Network Number of unique schools 33 Average number of students 7,942 Average number of students per school 310 Student race/ethnicity (%) American Indian/Alaska Native, non-Hispanic 0.1 Asian, non-Hispanic 0.2 Black, non-Hispanic 63.5 Hispanic 33.4 Native Hawaiian/Pacific Islander, non-Hispanic 0.0 White, non-Hispanic 2.0 Multiple race, non-Hispanic 0.6 No race/ethnicity available 0.0 Other student characteristics (%) Limited English proficiency 16.2 Special education 16.2 Free/reduced-price lunch eligible students 83.3 Statistics were computed for each school year then averaged across the three school years. Feasibility and Utility of the Tool Early feedback from the Network Specialist when using the Google Sheets version of the tool highlighted that given the extensive amount of interactions that the Specialist was having across the schools, the tool was critical in helping them keep track of the “wealth of information” that was being exchanged between the Specialist and each school. In addition, the Specialist noted that the log was critical in helping with annually revising the Network Specialist Implementation Guide, which is used to ensure fidelity of the Network Specialist intervention ( 8 ). However, prior to the conversion to the Google Form, the Specialist noted that “…It can get kind of tricky…that communications log can get kind of tricky for me, because I do interact a lot, … it can be tough just to make sure that I'm thinking about as many interactions as I have and trying to log them.” The early interviews also helped to identify that the Specialist was entering interactions with Network and OSHW staff that had to do with specific schools on separate Google Sheets related to the Specialist’s interaction with the Network or OSHW because someone from the Network or OSHW was also involved. That helped us realize that a lot of the interactions we had heard about from the Network Chief or OSHW staff were missing in our initial analyses of the log data using only the school tab in Google Sheets. We then went back and reviewed the entries from the other tabs to ensure that we did not miss any school-level interactions for our analyses. After the first full year of using the Google Form, the Network Specialist expressed that the tracking tool was feasible and useful, saying “the log is extensive, but doable. [It’s] something I have to carve out time to complete, but the data collection value makes it worth the effort and time in my opinion.” Completing the tracking tool also allowed the Network Specialist to provide more tailored support to schools, stating “it keeps me on track with what I'm working on and if I've been on top of my TA for all of my schools. It puts the TA work at the forefront of my mind for each school I'm interacting with.” The Specialist further noted that the log helps her keep a “record of the work [she’s] been doing with each school and hold myself accountable to continue TA with said schools.” And it enabled her to better keep track of the TA that she was (or was not) providing: “The TA log helps me to be intentional with my time and reflect on what TA I have or haven't been providing. If I see that I haven't logged a TA entry for a certain school or area in a while it alerts me that I need to make a move and get that TA up.” Importantly, CPS plans to continue using the tool for use by future Network Specialists. As noted by the Network Specialist, “Now that I've transitioned into a Team Lead position I want to use the TA log to monitor the work that the specialists I'm supervising are doing. It's a great way to collect data and monitor the progress of our specialists and know what work is being done in their network. I think this TA log also helps keep whoever is filling it out accountable and keep them on track to be providing the best TA possible.” Network Specialist Interactions and Costs The Network Specialist spent a total of 256.1 hours over the three SYs interacting with schools (SY 2020–2021: 41.6 hours; SY 2021–2022: 99.3 hours; SY 2022–2023: 115.2 hours), predominantly through emails, videochats, and in-person meetings (Table 2 ). As previously mentioned, interactions between the Network Specialist and schools were restricted by school leadership in the first year of the study due to COVID-19 (virtually all of the interactions in this year were by email). After prorating the costs for the Network Specialist’s interactions with schools, the intervention cost a total of $94,486 over the study period. The overwhelming majority (99.6%) of the intervention costs were derived from personnel expenses. This is in line with our intervention design that centered around the Network Specialist providing TA to schools. Only 0.2% ($224) of the costs derive from transportation, in part due to there being no on-site visits to schools during the first year of the intervention. Another 0.2% ($198) of the costs derive from equipment purchased. Table 2 Percentage of Network Specialist time spent and cost supporting schools by mode of interaction, tier of support provided, and SISTER implementation domains utilized. Duration of interactions, % Cost, $ Mode of interaction Email 39.8 37,605 Videochat 34.4 32,503 In-person 21.8 20,598 Phone 3.6 3,401 Not available 0.4 378 Tier of support Tier 1 26.7 25,228 Tier 2 31.7 29,952 Tier 3 41.4 39,117 Not available 0.2 189 SISTER implementation domains (multiple may apply) Use evaluative and iterative strategies 38.6 36,472 Train and educate stakeholders 36.8 34,771 Support educators 34.6 32,692 Develop stakeholder interrelationships 28.0 26,456 Provide interactive assistance 7.1 6,709 Engage consumers 2.0 1,890 Just over one-quarter (26.7%) of the Specialist’s time and cost ($25,228) was spent providing universal Tier 1 support (mean duration per interaction: 3.7 min [not shown in tables]), which included sending general information and resources to all schools. About one-third (31.7%) of the Specialist’s time and cost ($29,952) was spent providing more targeted Tier 2 support (mean: 14.5 min), which included responding to specific school requests for resources and connecting schools with specific services. The Network Specialist spent the highest proportion of time, 41.4%, and cost ($39,117) providing intensive Tier 3 support (mean: 40.3 min), which included regular check-in meetings with school leadership to discuss Healthy CPS achievement progress, set goals, and create plans of action. To support schools in implementing Healthy CPS policies, the Network Specialist used strategies that fell under the following SISTER domains: use evaluative and iterative strategies (38.6% of time spent, cost: $36,472); train and educate stakeholders (36.8%, cost: $34,771); support educators (34.6%, cost: $32,692); develop stakeholder interrelationships (28.0%, cost: $26,456); provide interactive assistance (7.1%, cost: $6,709); and engage consumers (2.0%, cost: $1,890). Three SISTER domains (adapt and tailor to context, use financial strategies, and change infrastructure) were not applicable to this study. Insert Table 2 about here As hypothesized, the Network Specialist’s tailoring of supports to schools was also reflected by the tracking tool. Schools with the highest duration of Tier 3 interactions with the Network Specialist had significantly lower average baseline SY2018-2019 Healthy CPS scores (score: 64.4) compared to schools with medium interactions (score: 67.3) and schools with low interactions (score: 78.4; p = 0.045; Table 3 ). In line with our hypothesis, our findings show that the Network Specialist targeted the most intensive Tier 3 supports to schools that most needed the support. There were no differences in baseline Healthy CPS scores by the duration of Tier 1 and Tier 2 supports provided. Table 3 Baseline Healthy CPS mean scores by duration of interactions for each tier of support. Baseline SY2018-2019 Healthy CPS score, mean Low duration interactions Medium duration interactions High duration interactions p-value Tier 1 support 77.1 64.7 69.6 0.090 Tier 2 support 69.7 77.4 64.3 0.092 Tier 3 support 78.4 67.3 64.4 0.045 Each cell represents the mean Healthy CPS baseline score (maximum score: 100) given the Tier of support and duration of interaction category. Interaction duration for Tier 1 support (low: 15–125 min; medium: 130-159.3 min; high: 159.5–207 min; N = 11 schools for each tertile), Tier 2 support (low: 5–90 min; medium: 93–184 min; high: 200–363 min; N = 11 for each tertile), and Tier 3 support (low: 0-135 min, N = 12; medium: 136–226 min, N = 11; high: 241–446 min, N = 10). P-values were computed using a Wald test of means within each tier of support. DISCUSSION Implementation science literature has long called for more detailed tracking of time and cost data associated with implementation activities ( 15 , 16 ). To our knowledge, this study is the first to use an ABTT tool in a school-based intervention to support health and wellness policy implementation. The tracking tool employed in this study was instrumental in describing the types and intensity of support the Network Specialist provided and the implementation strategies used. Without the use of the tracking tool, it would not have been possible to determine the duration and tiers of support provided to each school or to compute the proportion of the Network Specialist’s time spent on implementing the intervention compared to its design and planning. Our findings on the types of support provided and the implementation strategies used are useful for future researchers and policymakers in replicating and implementing this intervention with fidelity, in combination with the implementation guides produced as part of the larger intervention evaluation ( 8 ). In the study by Ritchie et al. which applied a similar tracking tool to assess the time and cost of facilitators implementing primary care mental health integrations, they found that different facilitators performed distinct implementation activities ( 21 ). Through the tracking tool, our study was able to show that the Network Specialist provided tailored supports to different schools and that the most intensive Tier 3 supports were targeted toward schools with the most need. This was achieved through the Network Specialist building close interrelationships with school leadership and providing targeted and specific TA according to each school’s needs. Though there are burdens and challenges to implementing an activity-based time tracking tool, it has been viewed as feasible to accomplish ( 33 ). However, intervention evaluations often do not elicit stakeholder perspectives on these tools in practice. In our study, the Network Specialist recounted the value of the tracking tool in providing continued and tailored supports to schools, particularly as the intervention is expanded to more schools. This highlights the importance of developing the tracking tool in close collaboration with the end user to maximize its usefulness. Through the community-engaged approach to develop the tracking tool and continuous feedback and improvements made to the tracking tool, we yielded a final product that is feasible to implement, valuable to both CPS and the end user, and sustainable as the tracking tool remains in use by CPS even outside of the evaluation period. As CPS expands support to additional schools and networks, the tracking tool can allow close monitoring of the TA provided by different Network Specialists. Limitations This study should be considered in light of the following limitations. First, the activity-based time tracking tool data were self-reported by the Network Specialist. However, the research team conducted continuous quality assurance of the data and discussed data and reporting standards with the Network Specialist on a regular basis to ensure that the tracking tool was used consistently and reliably. Second, the Network Specialist was restricted from contacting schools or providing in-person TA in the first year of the study due to COVID-19. This reduced the overall duration of interaction between the Network Specialist and schools and undercounts the intensive Tier 3 supports that would have otherwise been provided in year one. Third, there was a hiatus in the TA provided to schools between June to October of 2022 as the Network Specialist in the first two years of the study transitioned out of that role and a new Network Specialist was hired for year three of the study. This led to an undercounting of support that would have been provided. Conclusions In a school-based intervention to improve health and wellness policy compliance, an activity-based time tracking tool was used to measure the types and intensity of support provided to schools. In accordance with the intervention design, the tracking tool showed that the most intensive supports were targeted toward schools with the most need based on baseline policy compliance. The tracking tool was both feasible and valuable to the end user in providing tailored support. Our findings emphasize the need for utilizing similar tracking tools in school-based implementation science studies and in intervention evaluations. Abbreviations ABTT Activity-based time tracking CPS Chicago Public Schools FTE Full-time equivalent MTSS Multi-Tiered System of Supports OSHW Office of Student Health and Wellness SISTER School Implementation Strategies, Translating ERIC Resources SY School Year TA Technical assistance Declarations Ethics approval and consent to participate This study was approved by the University of Illinois Chicago Institutional Review Board (protocol #2019 − 1161) and the Chicago Public Schools Research Review Board (protocol #1582). Consent for publication Not applicable Competing interests The authors declare that they have no competing interests. Authors Information (Optional) Yu Chen Lin drafted and conducted this study while employed at the University of Illinois Chicago. Funding This study was supported by the Health Promotion and Disease Prevention Research Center cooperative agreement, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services (HHS) as part of a financial assistance award totaling $ 750,000. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, the U.S. Government, or the authors’ employers. Authors' contributions JFC conceptualized the study and obtained funding. YCL designed the data collection instruments and analysis plan. AW, CM, EJR, and MO collected the data. YCL and MO analyzed the data. YCL, MO, and JFC drafted the manuscript. JL and EJR provided feedback and substantive revisions. All authors read and approved the final manuscript. Acknowledgements Not applicable. Availability of data and materials The datasets generated and analyzed during the study are not publicly available due to the sensitive nature of the time and cost data but are available from the corresponding author on reasonable request. However, the instrument we used for time tracking is available upon request from the corresponding author. References Chriqui JF, Leider J, Temkin D, Piekarz-Porter E, Schermbeck RM, Stuart‐Cassel V. State laws matter when it comes to district policymaking relative to the whole school, whole community, whole child framework. J Sch Health. 2020;90(12):907–17. Chriqui JF, Leider J, Turner L, Piekarz-Porter E, Schwartz MB. State wellness policy requirement laws matter for district wellness policy comprehensiveness and wellness policy implementation in the United States. Nutrients. 2021;13(1):188. Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice. Prev Sci. 2019;20(6):914–35. Chicago Public Schools. Healthy CPS n.d. [ https://www.cps.edu/strategic-initiatives/healthy-cps/ Chicago Public Schools. Stats and Facts n.d. [ https://www.cps.edu/about/stats-facts/ Chicago Public Schools. Networks [ https://www.cps.edu/schools/networks/ Chicago Public Schools. School Data Chicago, IL: Chicago Public Schools; n.d. [ http://www.cps.edu/SCHOOLDATA/Pages/SchoolData.aspx Elizabeth Jarpe-Ratner MO, Sabrina Arancibia JF, Chriqui. Cassidy Malner, Kathryn Ramirez-Mercado, Jamie Tully, Tarrah DeClemente. Healthy CPS Network Specialist Implementation Guide [Internet]. Policy, Practice and Prevention Research Center, University of Illinois at Chicago, and Chicago Public Schools; 2023 [ https://indigo.uic.edu/articles/online_resource/Healthy_CPS_Network_Specialist_Implementation_Guide/25742859 Substance Abuse and Mental Health Services Administration. Focus on Prevention 2017 [ https://store.samhsa.gov/sites/default/files/sma10-4120.pdf Capacity Building Center for States. Working across the prevention continuum to strengthen families 2021 [ https://capacity.childwelfare.gov/sites/default/files/media_pdf/prevention-continuum-strengthen-families-cp-20119.pdf Sugai G, Horner RH. Responsiveness-to-intervention and school-wide positive behavior supports: Integration of multi-tiered system approaches. Exceptionality. 2009;17(4):223–37. Goodman S, Bohanon H. A framework for supporting all students: One-size-fits-all no longer works in schools. Am School Board J. 2018. Nitz J, Brack F, Hertel S, Krull J, Stephan H, Hennemann T et al. Multi-tiered systems of support with focus on behavioral modification in elementary schools: A systematic review. Heliyon. 2023. Bohnenkamp JH, Hartley SN, Splett JW, Halliday C, Collins D, Hoover S, et al. Promoting school safety through multi-tiered systems of support for student mental health. Preventing School Failure: Altern Educ Child Youth. 2023;67(1):9–17. Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ open. 2015;5(12):e009993. Wolfenden L, McCrabb S, Barnes C, O'Brien KM, Ng KW, Nathan NK et al. Strategies for enhancing the implementation of school-based policies or practices targeting diet, physical activity, obesity, tobacco or alcohol use. Cochrane Database Syst Reviews. 2022(8). Mogyorosy Z, Smith P. The main methodological issues in costing health care services: a literature review. Centre for Health Economics, University of York Working Papers. 2005(007cherp). Saldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The cost of implementing new strategies (COINS): a method for mapping implementation resources using the stages of implementation completion. Child Youth Serv Rev. 2014;39:177–82. Kaplan RS, Anderson SR. Time-driven activity-based costing: Børsen; 2008. Cidav Z, Mandell D, Pyne J, Beidas R, Curran G, Marcus S. A pragmatic method for costing implementation strategies using time-driven activity-based costing. Implement Sci. 2020;15:1–15. Ritchie MJ, Kirchner JE, Townsend JC, Pitcock JA, Dollar KM, Liu C-F. Time and organizational cost for facilitating implementation of primary care mental health integration. J Gen Intern Med. 2020;35:1001–10. Garcia CC, Bounthavong M, Gordon AJ, Gustavson AM, Kenny ME, Miller W, et al. Costs of implementing a multi-site facilitation intervention to increase access to medication treatment for opioid use disorder. Implement Sci Commun. 2023;4(1):91. Clinical and Translational Science Awards Consortium. Principles of community engagement, 2nd ed. Bethesda, MD: US Department of Health and Human Services, National Institutes of Health. 2011. https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf Key KD, Furr-Holden D, Lewis EY, Cunningham R, Zimmerman MA, Johnson-Lawrence V, et al. The continuum of community engagement in research: a roadmap for understanding and assessing progress. Progress community health partnerships: Res Educ action. 2019;13(4):427–34. Walsh-Bailey C, Palazzo LG, Jones SM, Mettert KD, Powell BJ, Wiltsey Stirman S, et al. A pilot study comparing tools for tracking implementation strategies and treatment adaptations. Implement Res Pract. 2021;2:26334895211016028. Spencer T, Rademaker L, Williams P, Loubier C. Online, asynchronous data collection in qualitative research. Popularizing Sch Research: Res Methods Practices. 2021:327. Finster MP, Feldman J. Cost-Effectiveness of 2 Support Models for a Healthy School Initiative. J Sch Health. 2020;90(9):724–30. Lane C, Nathan N, Reeves P, Sutherland R, Wolfenden L, Shoesmith A, et al. Economic evaluation of a multi-strategy intervention that improves school-based physical activity policy implementation. Implement Sci. 2022;17(1):40. Internal Revenue Service. Standard mileage rates [Internet]. [ https://www.irs.gov/tax-professionals/standard-mileage-rates Chicago Public Schools. School Profile Search [Internet]. [ https://www.cps.edu/schools/find-a-school/ Chicago Public Schools. School Year 2018–2019 Healthy CPS Alignment Report District Report 2020 [ https://drive.google.com/file/d/1y1Kkdb0zINbeJXK5_zhVt1_ilKeRCvN_/view Chicago Public Schools. Demographics [Internet]. [ https://www.cps.edu/about/district-data/demographics/ Levy DE, Singh D, Aschbrenner KA, Davies ME, Pelton-Cairns L, Kruse GR. Challenges and recommendations for measuring time devoted to implementation and intervention activities in health equity-focused, resource-constrained settings: a qualitative analysis. Implement Sci Commun. 2023;4(1):108. Supplementary Files AdditionalFIle1TATrackingLogExcerpt.pdf STROBEchecklistv4populated.pdf Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4707882","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":334973566,"identity":"a71cbd0e-fdf9-470f-b376-1c3c519fcd22","order_by":0,"name":"Yu Chen Lin","email":"","orcid":"","institution":"Moffitt Cancer Center","correspondingAuthor":false,"prefix":"","firstName":"Yu","middleName":"Chen","lastName":"Lin","suffix":""},{"id":334973567,"identity":"778d2beb-1fde-43a9-9c1a-69ebe551ec1f","order_by":1,"name":"Maddie Offstein","email":"","orcid":"","institution":"University of Illinois Chicago","correspondingAuthor":false,"prefix":"","firstName":"Maddie","middleName":"","lastName":"Offstein","suffix":""},{"id":334973568,"identity":"791e2757-a5a3-4a5f-997c-a090bba8e774","order_by":2,"name":"Cassidy Malner","email":"","orcid":"","institution":"CPS: Chicago Public Schools","correspondingAuthor":false,"prefix":"","firstName":"Cassidy","middleName":"","lastName":"Malner","suffix":""},{"id":334973569,"identity":"e8062888-3901-473b-8988-a9d0a62d4e11","order_by":3,"name":"Angel Williams","email":"","orcid":"","institution":"CPS: Chicago Public Schools","correspondingAuthor":false,"prefix":"","firstName":"Angel","middleName":"","lastName":"Williams","suffix":""},{"id":334973570,"identity":"7731551c-6d1e-42ab-aea2-994b70d7cb92","order_by":4,"name":"Elizabeth Jarpe-Ratner","email":"","orcid":"","institution":"University of Illinois Chicago","correspondingAuthor":false,"prefix":"","firstName":"Elizabeth","middleName":"","lastName":"Jarpe-Ratner","suffix":""},{"id":334973571,"identity":"9cac0540-1211-4826-9eba-ba0a764d3ac8","order_by":5,"name":"Julien Leider","email":"","orcid":"","institution":"University of Illinois Chicago","correspondingAuthor":false,"prefix":"","firstName":"Julien","middleName":"","lastName":"Leider","suffix":""},{"id":334973572,"identity":"3b69ed8b-a717-40bc-b4c8-a82cc98aa6f4","order_by":6,"name":"Jamie F Chriqui","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAtklEQVRIiWNgGAWjYBACxgYeBoYEBgYefiATyGcmQYtkA7FagIohlMEBMEWEFub2s8c+PCg7LGN8I7nxA0OFdWIDQYf15CXPSDh3mMfsRmKzBMOZdCK0zOAxZkhsA2tpY2BsO0yCFuMZIC3/SNFiIAHS0kCMlp4cY4aEc+k8EmceNkskHEs3JqjFsP2MMeOPMmt7/vb0hx8+1FjLEtYCVsEG5SUQUg4C8gzIWkbBKBgFo2AUYAMA4zM7NpOAjzoAAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0003-2538-0276","institution":"University of Illinois Chicago School of Public Health","correspondingAuthor":true,"prefix":"","firstName":"Jamie","middleName":"F","lastName":"Chriqui","suffix":""}],"badges":[],"createdAt":"2024-07-08 21:00:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4707882/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4707882/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":61639324,"identity":"028972fd-e9ed-4a4b-878c-37f754427c72","added_by":"auto","created_at":"2024-08-02 09:36:42","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":76208,"visible":true,"origin":"","legend":"\u003cp\u003eComponents of the Activity-Based Time Tracking Tool\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4707882/v1/937788a942d9e9e8b095a28c.png"},{"id":65151863,"identity":"24059762-4f62-4df8-90ce-22ed4db203b9","added_by":"auto","created_at":"2024-09-24 07:28:58","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":692483,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4707882/v1/fcdbb048-b4da-46bb-88be-c2767909fc85.pdf"},{"id":61639326,"identity":"ac8ec6d1-b769-489c-bc9f-79b4b975ca24","added_by":"auto","created_at":"2024-08-02 09:36:42","extension":"pdf","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":253072,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalFIle1TATrackingLogExcerpt.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4707882/v1/4c6b959cfb20b050230e94b7.pdf"},{"id":61639339,"identity":"53b014fe-6d3f-402d-9d83-c344be4a0239","added_by":"auto","created_at":"2024-08-02 09:36:43","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":163919,"visible":true,"origin":"","legend":"","description":"","filename":"STROBEchecklistv4populated.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4707882/v1/e5e28d7e29485b989ad7c2c9.pdf"}],"financialInterests":"","formattedTitle":"Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention","fulltext":[{"header":"Contributions to the literature ","content":"\u003cul\u003e\n \u003cli\u003eFew interventions assess the time and costs associated with implementation strategies used and activities performed, even more so in evaluating school-based interventions.\u003c/li\u003e\n \u003cli\u003eThe activity-based time tracking tool used in this study to support school-level health and wellness policy implementation revealed that the intervention targeted the most intensive supports to schools with the greatest need.\u003c/li\u003e\n \u003cli\u003eTime tracking tools should be developed in close collaboration with the end user to enhance their feasibility and usefulness.\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"BACKGROUND","content":"\u003cp\u003eIn the United States (U.S.), schools are required to comply with a multitude of health and wellness-related policies at the federal, state, and local (school district) levels (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Keeping track of all of the policies, much less implementing them, is complex and there is no one-size-fits-all approach. In fact, Cook et al.\u0026rsquo;s School Implementation Strategies, Translating ERIC Resources (SISTER) framework identified 79 strategies that could be used to support implementation of school-based interventions grouped across nine domains: (a) use evaluative and iterative strategies, (b) provide interactive assistance, (c) adapt and tailor to context, (d) develop stakeholder interrelationships, (e) train and educate stakeholders, (f) support educators, (g) engage consumers, (h) use financial strategies, and (i) change infrastructure (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eChicago Public Schools (CPS), one of the largest school districts in the U.S., struggled to support schools with implementation of over 50 federal, state, and local health and wellness-related policies. Recognizing this, in 2016, CPS created the Healthy CPS initiative to help schools navigate and implement these policies (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e). For the first four years of the Healthy CPS initiative, the district\u0026rsquo;s Office of Student Health and Wellness (OSHW) tried to support schools through standardized technical assistance (TA). However, it became clear that with over 600 schools (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e), centralized TA was not realistic and did not provide schools with the level of support they required. Instead, CPS\u0026rsquo; OSHW wanted to pilot test providing more localized support to schools through their 17 geographic networks; each network is comparable to a small-medium sized school district elsewhere in the country, with an average of more than 14,000 students per district and ~\u0026thinsp;15\u0026ndash;35 schools per network (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). Each network is led by a Network Chief and each network provides administrative support, strategic direction, and leadership development to schools within the network (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eWhen CPS schools need specific supports beyond the network, they are typically referred to offices and units throughout the district (e.g., Children and Family Benefits, OSHW, etc.). However, in 2020, in a collaborative community-academic partnership with the Policy, Practice and Prevention Research Center at the University of Illinois Chicago, CPS piloted an intervention to support one network in the west side of Chicago with implementing Healthy CPS. To implement this intervention, CPS\u0026rsquo; OSHW created a new Healthy CPS Network Specialist position to help schools implement these policies by providing TA and connecting schools to various resources (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). The Network Specialist used a Multi-Tiered System of Supports (MTSS) framework to provide schools with varying tiers of support in implementing Healthy CPS policies. Similar to the continuum of prevention model used to prevent substance use and to improve child welfare (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e), the MTSS framework is used by the education sector to provide \u0026ldquo;Tier 1\u0026rdquo; universal support to all schools/students, \u0026ldquo;Tier 2\u0026rdquo; targeted support to a subset of schools/students that require additional assistance, and \u0026ldquo;Tier 3\u0026rdquo; individualized support to schools/students with ongoing and intensive needs (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). School-based interventions delivered using a MTSS framework have been shown to improve student mental health, social behaviors, school engagement, academic performance, and more (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e). The Network Specialist\u0026rsquo;s activities were aligned with Cook et al.\u0026rsquo;s SISTER strategies (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). One of the goals with the Network Specialist intervention was to not only test the efficacy of the Network Specialist position in helping schools improve Healthy CPS compliance but also, equally important for the district leadership, to provide data for CPS to determine the nature of the supports provided by the Specialist, the time spent on providing supports by MTSS Tier, and the cost to do so.\u003c/p\u003e \u003cp\u003eHowever, detailed time and cost data on intervention activities are often lacking in implementation science, even more so in evaluating school-based interventions (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). Various methods have been developed to fill this gap. A common costing method is the top-down approach, which is one that allocates the overall time and cost spent at the organization level to the underlying implementation activities (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). This type of allocation method relies heavily on the richness and details of data on intervention activities to approximate or assume the distribution of time and cost (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). This approach often suffers from lack of accuracy, especially for evaluating interventions. Another costing method is the bottom-up approach, which aggregates individual-level data to derive the overall costs (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). This approach is often used in medical billing and fee-for-service applications, and is very time consuming and labor intensive (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). More modern and mixed approaches have been developed to overcome the challenges in these traditional methods. Saldana et al. developed a costing approach that mapped intervention costs to different stages of implementation (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). Based on a business accounting method (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e), Cidav et al. also detailed a time-driven activity-based costing approach that mapped costs to various implementation strategies (\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e). By using these approaches, researchers can directly compare the time and costs of various intervention activities performed and the implementation strategies utilized. Previous studies have used activity-based time tracking (ABTT) tools in primary care and substance use interventions to assess the cost of intervention activities (\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). These studies highlight the utility of these approaches in implementation science, and the use of time tracking tools helps collect data on intervention activities that inform decision-makers about how to implement policies and interventions with fidelity.\u003c/p\u003e \u003cp\u003eIn the \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003emethods\u003c/span\u003e section that follows, we describe the development of an ABTT tool for use by the Network Specialist in supporting schools with policy implementation. Then, in the results, we (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) document the utility and feasibility of using the tool; (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e) illustrate how the ABTT tool was used to assess the time and cost spent on providing supports by mode of support, MTSS Tier, and SISTER domain; and (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) show how the data from the ABTT tool were used to demonstrate that the Network Specialist provided support for schools with the greatest need. We hypothesized that the Network Specialist would spend a greater proportion of time providing intensive Tier 3 supports, and that those supports would predominantly be targeted toward schools with the most need as measured by baseline Healthy CPS policy compliance. To the best of our knowledge, an ABTT tool has not been used before in school-based interventions to support schools in implementing health and wellness policies.\u003c/p\u003e"},{"header":"METHODS","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eDevelopment of the Tool\u003c/h2\u003e \u003cdiv id=\"Sec4\" class=\"Section3\"\u003e \u003ch2\u003eUse of a Community-Engaged Process for Developing and Improving the Tool\u003c/h2\u003e \u003cp\u003eA community-engaged process (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) was taken to develop the ABTT tool, with iterative feedback from the Network Specialist and OSHW leadership to continuously improve the tracking tool for the end user (i.e., the Network Specialist) (\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e). Guided by existing literature on tracking implementation activities (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e), the initial version of the tracking tool used in school years (SY) 2020\u0026ndash;2022 was developed in Google Sheets, CPS\u0026rsquo; preferred platform, to capture the Network Specialist\u0026rsquo;s day-to-day activities because it could be easily integrated into CPS\u0026rsquo; existing TA tracking infrastructure housed in Google Sheets. The Network Specialist recorded each interaction with schools on a weekly basis (as close to real time as possible), including basic information about the nature of the interaction (e.g., date, names and roles of persons interacted with, mode of contact, duration). An open-ended text field was also provided for the Network Specialist to describe the topic and content of the interaction.\u003c/p\u003e \u003cp\u003eHowever, upon reviewing the tracking tool data in 2022, it became apparent that it was not accurately capturing the details on the specific supports/strategies being used by the Specialist, nor could it be used to identify the MTSS Tiers of support provided by the Specialist. Thus, through a series of four meetings between the UIC research team and the Network Specialist, we developed a revised tool in Google Forms (for use in SY 2022\u0026ndash;2023), similar to completing an electronic survey. The revised version of the tool was expanded to collect additional information on how TA support was delivered, including additional open-ended fields, close-ended questions on the most commonly provided support, and Healthy CPS topics discussed. This was done to standardize the tracking tool and to enhance its reliability. Standardization of tracking TA support across multiple end users was essential to the validity and sustainability of the tracking tool as CPS sought to expand its use to other CPS networks with the planned onboarding of additional Network Specialists. The use of the Google Form allowed for more streamlined data collection and reduced missing or inaccurate data through the use of forced responses and data validation. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e summarizes the main components of the tool presented in this study; Additional File 1 contains the specific questions and response options included in the tool for the items captured for this study. (A full-length version of the tool which captured specific Healthy CPS policy issues being addressed at each school is available upon request from the corresponding author.)\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section3\"\u003e \u003ch2\u003eInsert Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e about here\u003c/h2\u003e \u003cp\u003eEntries from both iterations of the ABTT tool were processed monthly by the UIC research team using Stata v.16 (StataCorp, College Station, TX, USA) to ensure data fidelity. Any inconsistencies or missing entries were reconciled with the Network Specialist via follow-up discussions and email communications. Summary statistics were also generated monthly and reviewed to monitor implementation fidelity and provide continuous quality improvement feedback to the Network Specialist on how to effectively use the tool. Between SY 2020\u0026ndash;2023, a total of 78 interactions (total duration: 57.0 hours) took place between the Network Specialist and the research team for the purposes of continuous quality improvement of the tracking tool (e.g., via meetings/emails to clarify what interactions were being had). Face validity of the fields in the tracking tool was assessed through discussions with the Network Specialist to evaluate that their understanding of what each field was asking for aligned with what the research team had intended.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section3\"\u003e \u003ch2\u003eDetermining the SISTER Domains and Tiers of Support used by the Network Specialist\u003c/h2\u003e \u003cp\u003eOpen-ended text fields, specifically questions 10 and 13 in the ABTT tool (see Additional File 1), were used by the Network Specialist to describe the topic and content of the interaction, resources shared, referrals made, and activities/strategies used to support schools\u0026rsquo; health and wellness policy implementation efforts, as well as the tier of MTSS support provided. This open-ended format was used as Walsh-Bailey et al. found that an open-ended tool to record implementation strategies was more acceptable and feasible compared to a more structured tool forcing respondents to select from pre-populated options (\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e). Researchers also wanted the Network Specialist to be able to provide as much detail as possible, so that the interactions could be later reviewed and coded by the research team.\u003c/p\u003e \u003cp\u003eTo code the open-ended text field data, entries were exported from the ABTT tool into an Excel spreadsheet, and columns were added to code the entries using SISTER implementation strategy domains and definitions put forth by Cook et al. (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). Using these initial code definitions for the implementation strategy domains, the open-ended field data were independently coded by three members of the research team (two lead investigators and one research associate). After this initial review of the entries and application of the SISTER domain codes, the research team held multiple meetings to further refine the SISTER domain code definitions to be more specific to the work of the Network Specialist and identified specific SISTER implementation strategies within each SISTER domain that the Network Specialist was using to help schools achieve Healthy CPS compliance (e.g., prepare and identify champions was identified as a strategy used by the Network Specialist and defined as preparing and identifying Wellness Champions to become leaders in implementing Healthy CPS in their schools). The research team then re-reviewed the initial application of SISTER domain codes and resolved any differences through discussion (and in consultation with the Network Specialist when additional details were needed about the nature of the logged interaction). After confirming agreement on the application of the SISTER domains, the research team then coded for the specific SISTER implementation strategies within each domain that mapped to the regular activities of the Network Specialist, and similarly met to discuss how the codes were being applied and rectify any disagreements in application. Investigators arrived at a final codebook that contained 6 SISTER domains and 15 SISTER strategies. For the sake of brevity and due to the limited number of total interactions coded, data in this study is presented at the implementation strategy domain-level (and not at the more specific implementation strategy-level).\u003c/p\u003e \u003cp\u003eIn addition to applying SISTER domain and strategy codes, the ABTT tool data were analyzed by designating the MTSS Tier provided by the Network Specialist. The MTSS categories included Tiers 1, 2, and 3\u0026mdash;with Tier 1 supports being the least involved supports that are provided to all schools and Tiers 2 and 3 representing more focused and the most targeted supports to schools, respectively, which were provided based on individual schools\u0026rsquo; needs and Healthy CPS attainment status. When completing the log, the Network Specialist designated which tier of support they thought the interaction fell under \u0026ndash; which the research team then reviewed when processing the data to ensure that the interaction was categorized correctly. The Network Specialist\u0026rsquo;s understanding of the differences in tiers was also periodically checked during process evaluation interviews throughout the study period. From the tool, 556 interactions were reviewed and only one interaction logged by the Network Specialist could not be categorized into any of the three tiers of support.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eDetermining the Utility and Feasibility of the Tool\u003c/h2\u003e \u003cp\u003eQualitative data were used to assess the utility and feasibility of using the ABTT tool. First, three key informant interviews between the Network Specialist and the UIC research team where the tool was discussed were held between 2021 and 2023. (Separate meetings were held with the Specialist to discuss tool development and refinement to ensure tool validity through continuous quality improvement of the data entry, but those meetings did not capture data on the use or feasibility of the tool.) Additionally, an asynchronous structured interview over email took place in May 2024 with the Network Specialist (one of the co-authors, CM) to gain insight into their perspective on the feasibility and usefulness of the tool (\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003cb\u003eData Sources for Examining the Implementation Costs and Whether the Specialist Supported Schools with the Most Need\u003c/b\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eCost Data\u003c/h2\u003e \u003cp\u003eConsistent with previous studies evaluating school-based interventions, costs were measured from an education sector perspective and included only the programmatic costs (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e). This perspective is most relevant to decision-makers interested in implementing the intervention. Costs were grouped into three categories: personnel, travel, and equipment. Personnel costs were prorated based on the proportion of time the Network Specialist spent interacting with schools (13.7% in SY 2020\u0026ndash;2021, 32.0% in SY 2021\u0026ndash;2022, and 51.5% in SY 2022\u0026ndash;2023), excluding their interactions with CPS, OSHW, and the research team that pertained to the design and planning of the intervention or the tracking tool. Travel and equipment costs were not prorated. Interactions between the Network Specialist and schools were relatively low in the first year due to the COVID-19 pandemic when Chicago schools were operating remotely and the Network Specialist was restricted in their capacity to interact with schools by the school district leadership. Personnel costs included the salary and benefits of the Network Specialist (1 full-time equivalent [FTE]) and the Healthy CPS Program Manager, who reported spending 10% of their time supporting the Network Specialist in implementing the intervention (0.1 FTE). Travel costs were derived from reimbursement forms submitted by the Network Specialist to CPS to cover their visits to schools to conduct on-site meetings with school leadership and provide TA. Mileage rates for travel were determined based on the standard mileage rate published by the Internal Revenue Service for business use (\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e). Equipment costs were self-reported and included the purchasing of CPS-issued electronic devices.\u003c/p\u003e \u003cdiv id=\"Sec9\" class=\"Section3\"\u003e \u003ch2\u003eHealthy CPS Scores\u003c/h2\u003e \u003cp\u003eTo assess the extent to which the Network Specialist was providing targeted supports to schools with the greatest need at baseline, publicly available Healthy CPS annual data from SY 2018\u0026ndash;2019 were compiled for each school through CPS\u0026rsquo; School Profile Search (\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e). Healthy CPS for that SY assessed 32 policies pertaining to four health and wellness areas: 1) chronic disease management (e.g., whether staff completed the chronic conditions training), 2) health instruction (e.g., whether schools provided the required physical, nutrition, and sexual health education curriculum), 3) school health leadership and environments (e.g., whether the school has a Wellness Champion and team to facilitate wellness policy implementation, whether the school offered daily recess), and 4) health services (e.g., whether the school participated in dental and vision exam programs) (\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e). The Healthy CPS were collected through district administrative data and schools\u0026rsquo; self-reported surveys. The SY 2018\u0026ndash;2019 Healthy CPS score, computed as the averaged percentage score in complying with policies in each of the four health and wellness areas, serves as a baseline measure of a school\u0026rsquo;s need for TA before the Network Specialist intervention.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003eAnalysis\u003c/h2\u003e \u003cdiv id=\"Sec11\" class=\"Section3\"\u003e \u003ch2\u003eQualitative Analysis\u003c/h2\u003e \u003cp\u003eTo determine feasibility and utility of the tool, the qualitative interview data and the asynchronous email data were analyzed by two study authors (MO, EJR) using MaxQDA Pro (VERBI GmbH, Berlin, Germany). The transcripts and email were analyzed to identify specific feedback from the Network Specialist about the tool and how it was being used. The qualitative data were organized according to broad categories of feasibility and utility. Most of the interview data focused on feasibility, particularly strengths and weaknesses of using the tool, whereas the final email exchange captured specific information on the utility or value of using the tool.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003eQuantitative Analyses\u003c/h2\u003e \u003cp\u003eABTT tool data were pulled from Google Sheets or Google Forms monthly between June 2020 and June 2023. Any missing or inconsistent data were resolved through discussions with the Network Specialist. If an activity pertained to multiple schools, the duration was viewed as the overall time it took to complete the activity across the schools; thus, the duration was recoded by splitting the time equally among the schools involved. For example, if the Network Specialist spent 10 minutes to draft and send an email blast to all intervention schools, the overall duration of that interaction would be divided and attributed equally to each of the schools. Cost by mode of interaction, tier of support, and SISTER implementation domains were computed by the percentage of time spent by the Network Specialist.\u003c/p\u003e \u003cp\u003eFor each tier of support, the schools served by the Specialist were categorized into tertiles (low, medium, and high) based on the total duration of interactions with the Network Specialist falling into the given tier. Mean SY 2018\u0026ndash;2019 Healthy CPS scores were computed within each tier of support and tertile of interaction duration. A Wald test was performed to determine the statistical significance of differences in mean scores within each tier of support. All analyses were performed in Stata v.17 (StataCorp, College Station, TX, USA).\u003c/p\u003e \u003c/div\u003e"},{"header":"RESULTS","content":"\u003cp\u003eBetween SY 2020\u0026ndash;2021 and SY 2022\u0026ndash;2023, the Network Specialist supported 33 unique schools with an average of 7,942 students per SY (Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e). The students in the intervention (Network Specialist) network were predominantly Black and Hispanic, most students were eligible for free or reduced-price lunch, and about one-sixth of the students had limited English proficiency and required special education (\u003cspan class=\"CitationRef\"\u003e32\u003c/span\u003e). This study was approved by the University of Illinois Chicago Institutional Review Board (protocol #2019\u0026thinsp;\u0026minus;\u0026thinsp;1161).\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab1\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eSchool and student characteristics of the intervention network, school years 2020\u0026ndash;2023.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e\u003cstrong\u003eCount or Mean\u003c/strong\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eNetwork\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eNumber of unique schools\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e33\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eAverage number of students\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e7,942\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eAverage number of students per school\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e310\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eStudent race/ethnicity (%)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eAmerican Indian/Alaska Native, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e0.1\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eAsian, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e0.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eBlack, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e63.5\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eHispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e33.4\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eNative Hawaiian/Pacific Islander, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e0.0\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eWhite, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e2.0\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eMultiple race, non-Hispanic\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e0.6\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eNo race/ethnicity available\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e0.0\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eOther student characteristics (%)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eLimited English proficiency\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e16.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eSpecial education\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e16.2\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"377\"\u003e\n\u003cp\u003eFree/reduced-price lunch eligible students\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"152\"\u003e\n\u003cp\u003e83.3\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"2\" width=\"529\"\u003e\n\u003cp\u003eStatistics were computed for each school year then averaged across the three school years.\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\n\u003cdiv id=\"Sec15\" class=\"Section3\"\u003e\n\u003ch2\u003eFeasibility and Utility of the Tool\u003c/h2\u003e\n\u003cp\u003eEarly feedback from the Network Specialist when using the Google Sheets version of the tool highlighted that given the extensive amount of interactions that the Specialist was having across the schools, the tool was critical in helping them keep track of the \u0026ldquo;wealth of information\u0026rdquo; that was being exchanged between the Specialist and each school. In addition, the Specialist noted that the log was critical in helping with annually revising the Network Specialist Implementation Guide, which is used to ensure fidelity of the Network Specialist intervention (\u003cspan class=\"CitationRef\"\u003e8\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eHowever, prior to the conversion to the Google Form, the Specialist noted that \u0026ldquo;\u0026hellip;It can get kind of tricky\u0026hellip;that communications log can get kind of tricky for me, because I do interact a lot, \u0026hellip; it can be tough just to make sure that I'm thinking about as many interactions as I have and trying to log them.\u0026rdquo; The early interviews also helped to identify that the Specialist was entering interactions with Network and OSHW staff that had to do with specific schools on separate Google Sheets related to the Specialist\u0026rsquo;s interaction with the Network or OSHW because someone from the Network or OSHW was also involved. That helped us realize that a lot of the interactions we had heard about from the Network Chief or OSHW staff were missing in our initial analyses of the log data using only the school tab in Google Sheets. We then went back and reviewed the entries from the other tabs to ensure that we did not miss any school-level interactions for our analyses.\u003c/p\u003e\n\u003cp\u003eAfter the first full year of using the Google Form, the Network Specialist expressed that the tracking tool was feasible and useful, saying \u0026ldquo;the log is extensive, but doable. [It\u0026rsquo;s] something I have to carve out time to complete, but the data collection value makes it worth the effort and time in my opinion.\u0026rdquo; Completing the tracking tool also allowed the Network Specialist to provide more tailored support to schools, stating \u0026ldquo;it keeps me on track with what I'm working on and if I've been on top of my TA for all of my schools. It puts the TA work at the forefront of my mind for each school I'm interacting with.\u0026rdquo; The Specialist further noted that the log helps her keep a \u0026ldquo;record of the work [she\u0026rsquo;s] been doing with each school and hold myself accountable to continue TA with said schools.\u0026rdquo; And it enabled her to better keep track of the TA that she was (or was not) providing: \u0026ldquo;The TA log helps me to be intentional with my time and reflect on what TA I have or haven't been providing. If I see that I haven't logged a TA entry for a certain school or area in a while it alerts me that I need to make a move and get that TA up.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eImportantly, CPS plans to continue using the tool for use by future Network Specialists. As noted by the Network Specialist, \u0026ldquo;Now that I've transitioned into a Team Lead position I want to use the TA log to monitor the work that the specialists I'm supervising are doing. It's a great way to collect data and monitor the progress of our specialists and know what work is being done in their network. I think this TA log also helps keep whoever is filling it out accountable and keep them on track to be providing the best TA possible.\u0026rdquo;\u003c/p\u003e\n\u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\n\u003ch2\u003eNetwork Specialist Interactions and Costs\u003c/h2\u003e\n\u003cp\u003eThe Network Specialist spent a total of 256.1 hours over the three SYs interacting with schools (SY 2020\u0026ndash;2021: 41.6 hours; SY 2021\u0026ndash;2022: 99.3 hours; SY 2022\u0026ndash;2023: 115.2 hours), predominantly through emails, videochats, and in-person meetings (Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e). As previously mentioned, interactions between the Network Specialist and schools were restricted by school leadership in the first year of the study due to COVID-19 (virtually all of the interactions in this year were by email). After prorating the costs for the Network Specialist\u0026rsquo;s interactions with schools, the intervention cost a total of $94,486 over the study period. The overwhelming majority (99.6%) of the intervention costs were derived from personnel expenses. This is in line with our intervention design that centered around the Network Specialist providing TA to schools. Only 0.2% ($224) of the costs derive from transportation, in part due to there being no on-site visits to schools during the first year of the intervention. Another 0.2% ($198) of the costs derive from equipment purchased.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab2\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003ePercentage of Network Specialist time spent and cost supporting schools by mode of interaction, tier of support provided, and SISTER implementation domains utilized.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e\u003cstrong\u003eDuration of interactions, %\u003c/strong\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e\u003cstrong\u003eCost, $\u003c/strong\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eMode of interaction\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eEmail\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e39.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e37,605\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eVideochat\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e34.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e32,503\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eIn-person\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e21.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e20,598\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003ePhone\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e3.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e3,401\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eNot available\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e0.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e378\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eTier of support\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eTier 1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e26.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e25,228\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eTier 2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e31.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e29,952\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eTier 3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e41.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e39,117\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eNot available\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e0.2\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e189\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eSISTER implementation domains (multiple may apply)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\u0026nbsp;\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eUse evaluative and iterative strategies\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e38.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e36,472\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eTrain and educate stakeholders\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e36.8\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e34,771\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eSupport educators\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e34.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e32,692\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eDevelop stakeholder interrelationships\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e28.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e26,456\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eProvide interactive assistance\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e7.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e6,709\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"378\"\u003e\n\u003cp\u003eEngage consumers\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"132\"\u003e\n\u003cp\u003e2.0\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd width=\"114\"\u003e\n\u003cp\u003e1,890\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003eJust over one-quarter (26.7%) of the Specialist\u0026rsquo;s time and cost ($25,228) was spent providing universal Tier 1 support (mean duration per interaction: 3.7 min [not shown in tables]), which included sending general information and resources to all schools. About one-third (31.7%) of the Specialist\u0026rsquo;s time and cost ($29,952) was spent providing more targeted Tier 2 support (mean: 14.5 min), which included responding to specific school requests for resources and connecting schools with specific services. The Network Specialist spent the highest proportion of time, 41.4%, and cost ($39,117) providing intensive Tier 3 support (mean: 40.3 min), which included regular check-in meetings with school leadership to discuss Healthy CPS achievement progress, set goals, and create plans of action.\u003c/p\u003e\n\u003cp\u003eTo support schools in implementing Healthy CPS policies, the Network Specialist used strategies that fell under the following SISTER domains: use evaluative and iterative strategies (38.6% of time spent, cost: $36,472); train and educate stakeholders (36.8%, cost: $34,771); support educators (34.6%, cost: $32,692); develop stakeholder interrelationships (28.0%, cost: $26,456); provide interactive assistance (7.1%, cost: $6,709); and engage consumers (2.0%, cost: $1,890). Three SISTER domains (adapt and tailor to context, use financial strategies, and change infrastructure) were not applicable to this study.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\n\u003ch2\u003eInsert Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e about here\u003c/h2\u003e\n\u003cp\u003eAs hypothesized, the Network Specialist\u0026rsquo;s tailoring of supports to schools was also reflected by the tracking tool. Schools with the highest duration of Tier 3 interactions with the Network Specialist had significantly lower average baseline SY2018-2019 Healthy CPS scores (score: 64.4) compared to schools with medium interactions (score: 67.3) and schools with low interactions (score: 78.4; p\u0026thinsp;=\u0026thinsp;0.045; Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e). In line with our hypothesis, our findings show that the Network Specialist targeted the most intensive Tier 3 supports to schools that most needed the support. There were no differences in baseline Healthy CPS scores by the duration of Tier 1 and Tier 2 supports provided.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n\u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n\u003ctable id=\"Tab3\" border=\"1\"\u003e\u003ccaption\u003e\n\u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n\u003cdiv class=\"CaptionContent\"\u003e\n\u003cp\u003eBaseline Healthy CPS mean scores by duration of interactions for each tier of support.\u003c/p\u003e\n\u003c/div\u003e\n\u003c/caption\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eBaseline SY2018-2019 Healthy CPS score, mean\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eLow duration interactions\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eMedium duration interactions\u003c/p\u003e\n\u003c/th\u003e\n\u003cth align=\"left\"\u003e\n\u003cp\u003eHigh duration interactions\u003c/p\u003e\n\u003c/th\u003e\n\u003cth colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003e\u003cem\u003ep-value\u003c/em\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTier 1 support\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e77.1\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e64.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e69.6\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003e\u003cem\u003e0.090\u003c/em\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTier 2 support\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e69.7\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e77.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e64.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003e\u003cem\u003e0.092\u003c/em\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003eTier 3 support\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e78.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e67.3\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd align=\"left\"\u003e\n\u003cp\u003e64.4\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd colspan=\"2\" align=\"left\"\u003e\n\u003cp\u003e\u003cem\u003e0.045\u003c/em\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd colspan=\"6\" align=\"left\"\u003e\n\u003cp\u003eEach cell represents the mean Healthy CPS baseline score (maximum score: 100) given the Tier of support and duration of interaction category. Interaction duration for Tier 1 support (low: 15\u0026ndash;125 min; medium: 130-159.3 min; high: 159.5\u0026ndash;207 min; N\u0026thinsp;=\u0026thinsp;11 schools for each tertile), Tier 2 support (low: 5\u0026ndash;90 min; medium: 93\u0026ndash;184 min; high: 200\u0026ndash;363 min; N\u0026thinsp;=\u0026thinsp;11 for each tertile), and Tier 3 support (low: 0-135 min, N\u0026thinsp;=\u0026thinsp;12; medium: 136\u0026ndash;226 min, N\u0026thinsp;=\u0026thinsp;11; high: 241\u0026ndash;446 min, N\u0026thinsp;=\u0026thinsp;10). P-values were computed using a Wald test of means within each tier of support.\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\u003c/div\u003e"},{"header":"DISCUSSION","content":"\u003cp\u003eImplementation science literature has long called for more detailed tracking of time and cost data associated with implementation activities (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). To our knowledge, this study is the first to use an ABTT tool in a school-based intervention to support health and wellness policy implementation. The tracking tool employed in this study was instrumental in describing the types and intensity of support the Network Specialist provided and the implementation strategies used. Without the use of the tracking tool, it would not have been possible to determine the duration and tiers of support provided to each school or to compute the proportion of the Network Specialist\u0026rsquo;s time spent on implementing the intervention compared to its design and planning. Our findings on the types of support provided and the implementation strategies used are useful for future researchers and policymakers in replicating and implementing this intervention with fidelity, in combination with the implementation guides produced as part of the larger intervention evaluation (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eIn the study by Ritchie et al. which applied a similar tracking tool to assess the time and cost of facilitators implementing primary care mental health integrations, they found that different facilitators performed distinct implementation activities (\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e). Through the tracking tool, our study was able to show that the Network Specialist provided tailored supports to different schools and that the most intensive Tier 3 supports were targeted toward schools with the most need. This was achieved through the Network Specialist building close interrelationships with school leadership and providing targeted and specific TA according to each school\u0026rsquo;s needs.\u003c/p\u003e \u003cp\u003eThough there are burdens and challenges to implementing an activity-based time tracking tool, it has been viewed as feasible to accomplish (\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e). However, intervention evaluations often do not elicit stakeholder perspectives on these tools in practice. In our study, the Network Specialist recounted the value of the tracking tool in providing continued and tailored supports to schools, particularly as the intervention is expanded to more schools. This highlights the importance of developing the tracking tool in close collaboration with the end user to maximize its usefulness. Through the community-engaged approach to develop the tracking tool and continuous feedback and improvements made to the tracking tool, we yielded a final product that is feasible to implement, valuable to both CPS and the end user, and sustainable as the tracking tool remains in use by CPS even outside of the evaluation period. As CPS expands support to additional schools and networks, the tracking tool can allow close monitoring of the TA provided by different Network Specialists.\u003c/p\u003e \u003cp\u003eLimitations\u003c/p\u003e \u003cp\u003eThis study should be considered in light of the following limitations. First, the activity-based time tracking tool data were self-reported by the Network Specialist. However, the research team conducted continuous quality assurance of the data and discussed data and reporting standards with the Network Specialist on a regular basis to ensure that the tracking tool was used consistently and reliably. Second, the Network Specialist was restricted from contacting schools or providing in-person TA in the first year of the study due to COVID-19. This reduced the overall duration of interaction between the Network Specialist and schools and undercounts the intensive Tier 3 supports that would have otherwise been provided in year one. Third, there was a hiatus in the TA provided to schools between June to October of 2022 as the Network Specialist in the first two years of the study transitioned out of that role and a new Network Specialist was hired for year three of the study. This led to an undercounting of support that would have been provided.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eIn a school-based intervention to improve health and wellness policy compliance, an activity-based time tracking tool was used to measure the types and intensity of support provided to schools. In accordance with the intervention design, the tracking tool showed that the most intensive supports were targeted toward schools with the most need based on baseline policy compliance. The tracking tool was both feasible and valuable to the end user in providing tailored support. Our findings emphasize the need for utilizing similar tracking tools in school-based implementation science studies and in intervention evaluations.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"609\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eABTT\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eActivity-based time tracking\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eCPS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eChicago Public Schools\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eFTE\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eFull-time equivalent\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eMTSS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eMulti-Tiered System of Supports\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eOSHW\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eOffice of Student Health and Wellness\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\"\u003e\n \u003cp\u003eSISTER\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\"\u003e\n \u003cp\u003eSchool Implementation Strategies, Translating ERIC Resources\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eSY\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eSchool Year\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"17.24137931034483%\" valign=\"bottom\"\u003e\n \u003cp\u003eTA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"82.75862068965517%\" valign=\"bottom\"\u003e\n \u003cp\u003eTechnical assistance\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e"},{"header":"Declarations","content":" \u003cp\u003e \u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e \u003cp\u003eThis study was approved by the University of Illinois Chicago Institutional Review Board (protocol #2019\u0026thinsp;\u0026minus;\u0026thinsp;1161) and the Chicago Public Schools Research Review Board (protocol #1582).\u003c/p\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eConsent for publication\u003c/strong\u003e \u003cp\u003eNot applicable\u003c/p\u003e \u003c/p\u003e\u003cp\u003e \u003ch2\u003eCompeting interests\u003c/h2\u003e \u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e \u003c/p\u003e\u003cp\u003e \u003ch2\u003eAuthors Information (Optional)\u003c/h2\u003e \u003cp\u003eYu Chen Lin drafted and conducted this study while employed at the University of Illinois Chicago.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eFunding\u003c/h2\u003e \u003cp\u003eThis study was supported by the Health Promotion and Disease Prevention Research Center cooperative agreement, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services (HHS) as part of a financial assistance award totaling \u003cspan\u003e$\u003c/span\u003e750,000. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement, by CDC/HHS, the U.S. Government, or the authors\u0026rsquo; employers.\u003c/p\u003e\u003ch2\u003eAuthors' contributions\u003c/h2\u003e \u003cp\u003eJFC conceptualized the study and obtained funding. YCL designed the data collection instruments and analysis plan. AW, CM, EJR, and MO collected the data. YCL and MO analyzed the data. YCL, MO, and JFC drafted the manuscript. JL and EJR provided feedback and substantive revisions. All authors read and approved the final manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgements\u003c/h2\u003e \u003cp\u003eNot applicable.\u003c/p\u003e\u003ch2\u003eAvailability of data and materials\u003c/h2\u003e \u003cp\u003eThe datasets generated and analyzed during the study are not publicly available due to the sensitive nature of the time and cost data but are available from the corresponding author on reasonable request. However, the instrument we used for time tracking is available upon request from the corresponding author.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eChriqui JF, Leider J, Temkin D, Piekarz-Porter E, Schermbeck RM, Stuart‐Cassel V. State laws matter when it comes to district policymaking relative to the whole school, whole community, whole child framework. J Sch Health. 2020;90(12):907\u0026ndash;17.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChriqui JF, Leider J, Turner L, Piekarz-Porter E, Schwartz MB. State wellness policy requirement laws matter for district wellness policy comprehensiveness and wellness policy implementation in the United States. Nutrients. 2021;13(1):188.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice. Prev Sci. 2019;20(6):914\u0026ndash;35.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. Healthy CPS n.d. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cps.edu/strategic-initiatives/healthy-cps/\u003c/span\u003e\u003cspan address=\"https://www.cps.edu/strategic-initiatives/healthy-cps/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. Stats and Facts n.d. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cps.edu/about/stats-facts/\u003c/span\u003e\u003cspan address=\"https://www.cps.edu/about/stats-facts/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. Networks [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cps.edu/schools/networks/\u003c/span\u003e\u003cspan address=\"https://www.cps.edu/schools/networks/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. School Data Chicago, IL: Chicago Public Schools; n.d. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.cps.edu/SCHOOLDATA/Pages/SchoolData.aspx\u003c/span\u003e\u003cspan address=\"http://www.cps.edu/SCHOOLDATA/Pages/SchoolData.aspx\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eElizabeth Jarpe-Ratner MO, Sabrina Arancibia JF, Chriqui. Cassidy Malner, Kathryn Ramirez-Mercado, Jamie Tully, Tarrah DeClemente. Healthy CPS Network Specialist Implementation Guide [Internet]. Policy, Practice and Prevention Research Center, University of Illinois at Chicago, and Chicago Public Schools; 2023 [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://indigo.uic.edu/articles/online_resource/Healthy_CPS_Network_Specialist_Implementation_Guide/25742859\u003c/span\u003e\u003cspan address=\"https://indigo.uic.edu/articles/online_resource/Healthy_CPS_Network_Specialist_Implementation_Guide/25742859\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSubstance Abuse and Mental Health Services Administration. Focus on Prevention 2017 [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://store.samhsa.gov/sites/default/files/sma10-4120.pdf\u003c/span\u003e\u003cspan address=\"https://store.samhsa.gov/sites/default/files/sma10-4120.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCapacity Building Center for States. Working across the prevention continuum to strengthen families 2021 [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://capacity.childwelfare.gov/sites/default/files/media_pdf/prevention-continuum-strengthen-families-cp-20119.pdf\u003c/span\u003e\u003cspan address=\"https://capacity.childwelfare.gov/sites/default/files/media_pdf/prevention-continuum-strengthen-families-cp-20119.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSugai G, Horner RH. Responsiveness-to-intervention and school-wide positive behavior supports: Integration of multi-tiered system approaches. Exceptionality. 2009;17(4):223\u0026ndash;37.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoodman S, Bohanon H. A framework for supporting all students: One-size-fits-all no longer works in schools. Am School Board J. 2018.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNitz J, Brack F, Hertel S, Krull J, Stephan H, Hennemann T et al. Multi-tiered systems of support with focus on behavioral modification in elementary schools: A systematic review. Heliyon. 2023.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBohnenkamp JH, Hartley SN, Splett JW, Halliday C, Collins D, Hoover S, et al. Promoting school safety through multi-tiered systems of support for student mental health. Preventing School Failure: Altern Educ Child Youth. 2023;67(1):9\u0026ndash;17.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care\u0026mdash;effectiveness of strategies for improving implementation of complex interventions: systematic review of reviews. BMJ open. 2015;5(12):e009993.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWolfenden L, McCrabb S, Barnes C, O'Brien KM, Ng KW, Nathan NK et al. Strategies for enhancing the implementation of school-based policies or practices targeting diet, physical activity, obesity, tobacco or alcohol use. Cochrane Database Syst Reviews. 2022(8).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMogyorosy Z, Smith P. The main methodological issues in costing health care services: a literature review. Centre for Health Economics, University of York Working Papers. 2005(007cherp).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSaldana L, Chamberlain P, Bradford WD, Campbell M, Landsverk J. The cost of implementing new strategies (COINS): a method for mapping implementation resources using the stages of implementation completion. Child Youth Serv Rev. 2014;39:177\u0026ndash;82.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKaplan RS, Anderson SR. Time-driven activity-based costing: B\u0026oslash;rsen; 2008.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCidav Z, Mandell D, Pyne J, Beidas R, Curran G, Marcus S. A pragmatic method for costing implementation strategies using time-driven activity-based costing. Implement Sci. 2020;15:1\u0026ndash;15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRitchie MJ, Kirchner JE, Townsend JC, Pitcock JA, Dollar KM, Liu C-F. Time and organizational cost for facilitating implementation of primary care mental health integration. J Gen Intern Med. 2020;35:1001\u0026ndash;10.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGarcia CC, Bounthavong M, Gordon AJ, Gustavson AM, Kenny ME, Miller W, et al. Costs of implementing a multi-site facilitation intervention to increase access to medication treatment for opioid use disorder. Implement Sci Commun. 2023;4(1):91.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eClinical and Translational Science Awards Consortium. Principles of community engagement, 2nd ed. Bethesda, MD: US Department of Health and Human Services, National Institutes of Health. 2011. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf\u003c/span\u003e\u003cspan address=\"https://www.atsdr.cdc.gov/communityengagement/pdf/PCE_Report_508_FINAL.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKey KD, Furr-Holden D, Lewis EY, Cunningham R, Zimmerman MA, Johnson-Lawrence V, et al. The continuum of community engagement in research: a roadmap for understanding and assessing progress. Progress community health partnerships: Res Educ action. 2019;13(4):427\u0026ndash;34.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWalsh-Bailey C, Palazzo LG, Jones SM, Mettert KD, Powell BJ, Wiltsey Stirman S, et al. A pilot study comparing tools for tracking implementation strategies and treatment adaptations. Implement Res Pract. 2021;2:26334895211016028.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSpencer T, Rademaker L, Williams P, Loubier C. Online, asynchronous data collection in qualitative research. Popularizing Sch Research: Res Methods Practices. 2021:327.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFinster MP, Feldman J. Cost-Effectiveness of 2 Support Models for a Healthy School Initiative. J Sch Health. 2020;90(9):724\u0026ndash;30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLane C, Nathan N, Reeves P, Sutherland R, Wolfenden L, Shoesmith A, et al. Economic evaluation of a multi-strategy intervention that improves school-based physical activity policy implementation. Implement Sci. 2022;17(1):40.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eInternal Revenue Service. Standard mileage rates [Internet]. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.irs.gov/tax-professionals/standard-mileage-rates\u003c/span\u003e\u003cspan address=\"https://www.irs.gov/tax-professionals/standard-mileage-rates\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. School Profile Search [Internet]. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cps.edu/schools/find-a-school/\u003c/span\u003e\u003cspan address=\"https://www.cps.edu/schools/find-a-school/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. School Year 2018\u0026ndash;2019 Healthy CPS Alignment Report District Report 2020 [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://drive.google.com/file/d/1y1Kkdb0zINbeJXK5_zhVt1_ilKeRCvN_/view\u003c/span\u003e\u003cspan address=\"https://drive.google.com/file/d/1y1Kkdb0zINbeJXK5_zhVt1_ilKeRCvN_/view\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChicago Public Schools. Demographics [Internet]. [ \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cps.edu/about/district-data/demographics/\u003c/span\u003e\u003cspan address=\"https://www.cps.edu/about/district-data/demographics/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLevy DE, Singh D, Aschbrenner KA, Davies ME, Pelton-Cairns L, Kruse GR. Challenges and recommendations for measuring time devoted to implementation and intervention activities in health equity-focused, resource-constrained settings: a qualitative analysis. Implement Sci Commun. 2023;4(1):108.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Activity-based time tracking, school health and wellness policy, implementation science, costs, community-engaged research","lastPublishedDoi":"10.21203/rs.3.rs-4707882/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4707882/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e \u003cb\u003eBackground.\u003c/b\u003e Detailed time and cost data are often lacking in implementation science, particularly in school-based interventions. In a pilot intervention in one Chicago Public Schools\u0026rsquo; geographic network, a Network Specialist was hired to provide schools with tailored technical assistance (TA) to support compliance with over 50 health-related policies (the Healthy CPS initiative). This study describes the methods for developing and implementing an activity-based time tracking tool to assess the Network Specialist\u0026rsquo;s fidelity, time, and cost in providing TA using a Multi-Tiered System of Supports framework (\u0026ldquo;Tier 1\u0026rdquo; universal support, \u0026ldquo;Tier 2\u0026rdquo; targeted support, and \u0026ldquo;Tier 3\u0026rdquo; intensive, individualized support).\u003c/p\u003e \u003cp\u003e \u003cb\u003eMethods.\u003c/b\u003e The tool was developed in close collaboration with the Network Specialist to capture the Specialist\u0026rsquo;s interactions with schools between 2020\u0026ndash;2023. Key informant interviews and asynchronous post-hoc feedback were qualitatively analyzed to assess the Specialist\u0026rsquo;s feedback on the tool. Descriptive statistics on school interactions, tiers of support provided, and domains of implementation support provided using the SISTER implementation science framework were generated from the tracking tool data. Differences in mean baseline Healthy CPS policy compliance based on the extent of schools\u0026rsquo; interactions with the Specialist in each tier of support were computed using Wald tests.\u003c/p\u003e \u003cp\u003e \u003cb\u003eResults.\u003c/b\u003e The Specialist described the tracking tool as feasible and useful in providing tailored support and advocated for its continued use as the intervention is expanded to additional networks. The Specialist spent the highest proportion of time and costs (41.4%, \u003cspan\u003e$\u003c/span\u003e39,117) providing intensive Tier 3 supports, and those supports were targeted toward schools with the most need. Schools receiving the most Tier 3 supports had lower baseline Healthy CPS compliance of 64.4%, versus 78.4% and 67.3% for schools receiving low and medium levels of Tier 3 supports, respectively (p-value\u0026thinsp;=\u0026thinsp;0.045).\u003c/p\u003e \u003cp\u003e \u003cb\u003eConclusions.\u003c/b\u003e Expanded use of time and cost tracking is needed in implementation science, particularly for school-based interventions. Time tracking tools help collect data on intervention activities that inform decision-makers about how to implement interventions with fidelity. Our findings point to the value of using a collaborative, partner-engaged approach to developing the tracking tool with the end user to maximize its feasibility, usefulness, utilization, and sustainability.\u003c/p\u003e","manuscriptTitle":"Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-08-02 09:36:31","doi":"10.21203/rs.3.rs-4707882/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"125ca94e-7cc1-4d7a-89a7-a91f4e88fcf7","owner":[],"postedDate":"August 2nd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-09-24T07:20:51+00:00","versionOfRecord":[],"versionCreatedAt":"2024-08-02 09:36:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4707882","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4707882","identity":"rs-4707882","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}