Use of an activity-based time tracking tool to support implementation of a school district-level technical assistance intervention

Abstract

Background. Detailed time and cost data are often lacking in implementation science, particularly in school-based interventions. In a pilot intervention in one Chicago Public Schools’ geographic network, a Network Specialist was hired to provide schools with tailored technical assistance (TA) to support compliance with over 50 health-related policies (the Healthy CPS initiative). This study describes the methods for developing and implementing an activity-based time tracking tool to assess the Network Specialist’s fidelity, time, and cost in providing TA using a Multi-Tiered System of Supports framework (“Tier 1” universal support, “Tier 2” targeted support, and “Tier 3” intensive, individualized support). Methods. The tool was developed in close collaboration with the Network Specialist to capture the Specialist’s interactions with schools between 2020–2023. Key informant interviews and asynchronous post-hoc feedback were qualitatively analyzed to assess the Specialist’s feedback on the tool. Descriptive statistics on school interactions, tiers of support provided, and domains of implementation support provided using the SISTER implementation science framework were generated from the tracking tool data. Differences in mean baseline Healthy CPS policy compliance based on the extent of schools’ interactions with the Specialist in each tier of support were computed using Wald tests. Results. The Specialist described the tracking tool as feasible and useful in providing tailored support and advocated for its continued use as the intervention is expanded to additional networks. The Specialist spent the highest proportion of time and costs (41.4%, $39,117) providing intensive Tier 3 supports, and those supports were targeted toward schools with the most need. Schools receiving the most Tier 3 supports had lower baseline Healthy CPS compliance of 64.4%, versus 78.4% and 67.3% for schools receiving low and medium levels of Tier 3 supports, respectively (p-value = 0.045). Conclusions. Expanded use of time and cost tracking is needed in implementation science, particularly for school-based interventions. Time tracking tools help collect data on intervention activities that inform decision-makers about how to implement interventions with fidelity. Our findings point to the value of using a collaborative, partner-engaged approach to developing the tracking tool with the end user to maximize its feasibility, usefulness, utilization, and sustainability.