Defining Success for Translational Research Organizations

Abstract

Translational research organizations (TROs) seek to enhance the clinical impact of scienti c discoveries and play a multifaceted role characterized by multidisciplinary collaboration, outreach initiatives, and the provision of shared resources and facilities. Given this complexity, TROs require a exible framework for performance assessment that tracks their progress, incentivizes fruitful activities, and aligns individuals throughout the organization. We suggest a framework that assesses TRO performance along seven main dimensions-funding, talent, creation, validation, dissemination, external uptake, and collaboration-and we encourage individual organizations to develop additional metrics as needed.

Measuring research impact in Australia’s medical research institutes: a scoping literature review of the objectives for and an assessment of the capabilities of research impact assessment frameworks

“…This implementation demonstrates the capacity for many similar frameworks, such as Payback, the Balanced Scorecard, the Translational Research Organizations Performance Model, the Comprehensive Research Metrics model, the ITHS KLM–WHO model, the RE-AIM model and REF to develop this capability, given their shared characteristics [28, 42, 44, 45, 54, 55]. …”

“…Indicators, self-assessed surveyYesYesYesYesPartialYesPossiblePossibleResearch Utilization LadderLandry, Amara [36]T3, T4Self-assessed surveyNoPartialNoPossibleNoPartial/noPartial/noPossibleSocietal Impact FrameworkMostert, Ellenbroek [52], Van Ark G [58]T3, T4Quant. IndicatorsPartialYesYesPossibleNoYesNoPossibleTelethon Kids Institute Research Impact FrameworkTelethon Kids Institute [46]T1–T4Mixed methodsYesYesYesPossibleYesYesPossiblePossibleTranslational Research Organizations Performance ModelPozen and Kline [44]T1–T4Quant. IndicatorsYesYesYesPossiblePartialYesYesYesWeiss Logic ModelWeiss [49]T1–T4Mixed methodsYesYesYesPossibleYesYesPossiblePossible

a Utilises the Khoury, Gwinn [37] definition of T1–T4

Yes/No – Met/Did not meet specified criteria; Partial – Meets criteria, but incomplete representation; Possible – With adaptions, capable of meeting criteria

…”

“…Research Utilization Framework, the Australian Research Council’s Excellence in Research Australia (ARC ERA) impact measures 1. The form of this information varies across peer-reviewed qualitative studies of impact [36, 39, 40], exclusively quantitative analyses [26, 33, 34, 41–44] and numerous mixed method assessments [5, 21, 27, 45–47]. …”

“…The majority of frameworks met this criteria, albeit that the emphasis varies from the relative impact of research outputs [5, 44, 45, 51] to organisational aims, such as progress towards strategic goals [32, 53]. While the methodologies adopted within the frameworks varied, most provide the grounds to facilitate comparison, albeit that greater transparency assists comparison in the “ doing, diffusion and impact of research within and across research fields ” [40].…”

“…The latter two are, however, limited by their focus solely upon final end-users of research. If imperatives, such as the lag to final impact [5, 28, 44] and sub-optimal research practices within HMR [14–17], are to be addressed, then a framework requires a broader interpretation of steps in the process of research translation. The Balanced Scorecard represents another performance management framework that provides for steps along the pathway to impact, but fails to guide the critical link to usage at each step [42].…”