Heterogeneity at Work

Abstract

The Clinical and Translational Science Award (CTSA) program is an ambitious multibillion dollar initiative sponsored by the National Institutes of Health (NIH) organized around the mission of facilitating the improved quality, efficiency, and effectiveness of translational health sciences research across the country. Although the NIH explicitly requires internal evaluation, funded CTSA institutions are given wide latitude to choose the structure and methods for evaluating their local CTSA program. The National Evaluators Survey was developed by a peer-led group of local CTSA evaluators as a voluntary effort to understand emerging differences and commonalities in evaluation teams and techniques across the 61 CTSA institutions funded nationwide. This article presents the results of the 2012 National Evaluators Survey, finding significant heterogeneity in evaluation staffing, organization, and methods across the 58 CTSAs institutions responding. The variety reflected in these findings represents both a liability and strength. A lack of standardization may impair the ability to make use of common metrics, but variation is also a successful evolutionary response to complexity. Additionally, the peer-led approach and simple design demonstrated by the questionnaire itself has value as an example of an evaluation technique with potential for replication in other areas across the CTSA institutions or any large-scale investment where multiple related teams across a wide geographic area are given the latitude to develop specialized approaches to fulfilling a common mission.

Advancing medical technology innovation and clinical translation via a model of industry-enabled technical and educational support: Indiana Clinical and Translational Sciences Institute’s Medical Technology Advance Program

“…In 2000, Balas and Boren [9] reported an average translation time of 17 years from biomedical research to clinical practice, prompting research into understanding the challenges to translation and identifying potential solutions. A report of the National Evaluators Survey, established by a coalition of CTSA evaluators, studied the implementation of various methods for the evaluation of advancing translation by the over 60 CTSA programs in 2012 and found significant variation in strategies and methods across programs [10]. Analysis has indicated a significant shortfall of programs providing regulatory education, which has proven increasingly deleterious for rapidly expanding translational research programs with limited access to this specialized training [11].…”

Opportunities for strengthening CTSA evaluation

“…Building off surveys described in ref. [22], in 2018, a workgroup established under the CTSA Program Evaluators Group administered a survey to 64 CTSA hub evaluation leaders to learn more about how CTSA hubs perceive and engage in both local and national evaluations [22]. Sixty-one CTSA hubs provided a response to the survey (95%).…”

Defining and Measuring Community Engagement and Community-Engaged Research: Clinical and Translational Science Institutional Practices

“…2. Although CTSA institutions function within unique local contexts and pursue distinct site priorities in terms of communities to engage and for what purpose, 24 many CTSA institutions define community engagement similarly. Even when definitions differ, similar programmatic characteristics are evident in the alignment of partners, stakeholders and Community Advisory Board members.…”