Strengthening the Science and Practice of Implementation Support: Evaluating the Effectiveness of Training and Technical Assistance Centers

Recent implementation science frameworks highlight the role of training and technical assistance (TTA) in building workforce capacity to implement evidence-based practices (EBPs). However, evaluation of TTA is limited. We describe three case examples that highlight TTA by three regional centers in the national Mental Health Technology Transfer Center (MHTTC) network. Each MHTTC formed Learning Communities (LCs) to facilitate connections among behavioral health professionals with the goals of sharing implementation strategies, discussing best-practices, and developing problem solving techniques. Data on outcomes were collected through a combination of self-report surveys and qualitative interviews. LC participants reported strong connectedness, gains in knowledge and skills, improvements in implementation capacity, and intentions to advocate for organizational and systems-level change. Furthermore, across the case examples, we identified LC characteristics that are associated with participant perceptions of outcomes, including tailoring LC content to workforce needs, providing culturally relevant information, engaging leaders, forming connections among participants and trainers, and challenging participants’ current workplace practices. These findings are interpreted through the lens of the Interactive Systems Framework, which focuses on how TTA, such as LCs, can facilitate connections between the theoretical and empirical foundations of interventions and the practices of implementing interventions in real-world settings to advance workforce capacity.

Supporting Implementation Through Online Learning Communities: Lessons Learned From a National Training and Technical Assistance Network

Olson,

Walker,

Chwastiak

et al. 2024

Technical Assistance for Systemic Change: Lessons Learned From a National Technical Assistance Center

Ward,

Farmer,

Livet

2024

Despite the millions of dollars awarded annually by the United States Department of Education to build implementation capacity through technical assistance (TA), data on TA effectiveness are severely lacking. Foundational to the operationalization and consistent research on TA effectiveness is the development and use of standardized TA core competencies, practices, and structures. Despite advances toward a consistent definition of TA, a gap still exists in understanding how these competencies are used within an operationalized set of TA practices to produce targeted outcomes at both individual and organizational levels to facilitate implementation of evidence-based practices. The current article describes key insights derived from the evaluation of an operationalized set of TA practices used by a nationally funded TA center, the State Implementation & Scaling Up of Evidence Based Practices (SISEP) Center. The TA provided by the Center supports the uptake of evidence-based practices in K-12 education for students with disabilities. Lessons learned include: (1) the need to understand the complexities and dependencies of operationalizing TA both longitudinally and at multiple levels of the system (state, regional, local); (2) the relative importance of building general and innovation-specific capacity for implementation success; (3) the value of using a co-design and participatory approach for effective TA delivery; (4) the need to develop TA providers’ educational and implementation fluency across areas and levels of the system receiving TA; and (5) the need to ensure coordination and alignment of TA providers from different centers. Gaining an understanding into optimal TA practices will not only provide clarity of definition fundamental to TA research, but it will also inform the conceptual framing and practice of TA.

The Technical Assistance (TA) Effectiveness Logic Model: A Tool for Systematically Planning, Delivering, and Evaluating TA

Scott,

Chagnon,

Wandersman

2024

Technical assistance (TA) is a tailored approach to capacity building that has grown in use across diverse settings over the past two decades, particularly in the domains of health improvement, child welfare, youth development, and education. In practice, TA services often include a combination of activities (e.g., coaching, professional development, site visits, and resource sharing) and vary in dosage, modality, and numerous other dimensions. While tailoring to recipient needs holds significant value, the individualized nature of TA creates challenges for assessing the effectiveness of TA. The difficulty of determining what contributes to TA outcomes is amplified when TA delivery is largely reactive (rather than proactive) and unaccompanied by a systematic approach to planning, delivery, and evaluation. Logic models are a well-established tool and a straightforward way to make TA more systematic. In this article, we introduce the TA Effectiveness Logic Model as a skeletal framework to guide systematic TA planning, delivery, and evaluation. The TA Effectiveness Logic Model graphically presents the rationale and expectations for how TA works in a setting. We describe two types of TA effectiveness logic models: basic and contextual. In addition, we offer a case example from work based on a national training and TA center to illustrate both types of logic models. Lastly, we discuss practical implications of TA effectiveness logic models for TA collaborators (funders, training and TA center administrators, TA providers, TA recipients, and researchers). Routinizing the use of TA effectiveness logic models has benefits for both the science and practice of TA.