Background: The Special Programme for Research and Training in Tropical Diseases developed a massive open online course (MOOC) on implementation research with a focus on infectious diseases of poverty to reinforce the explanation of implementation research concepts through real case studies. Following a pilot in 2017, two cohorts undertook the nalised version of the implementation research MOOC in 2018. Methods: This study evaluates the 2018 MOOC learners' knowledge in implementation research after completing the MOOC. Learners were invited to complete an anonymous exit survey on completion of the MOOC, regardless of whether or not they had received a certi cate of completion. Results: Of the almost 4,000 enrolled in the two sessions of the MOOC, about 30% completed all ve modules and the assessments and were and awarded a certi cate. The majority of the participants were early to mid-career professionals, under the age of 40, and from low-and middle-income countries. They were slightly more likely to be men (56%) with a Bachelor or a Master's degree. Participants were public health researchers (45%), public health o cers (11%) or students (11%). On completion of the course, an exit survey revealed that 80.9% of respondents indicated signi cant improvement to strong and very strong implementation research knowledge. Conclusions: This evaluation clearly shows the usefulness of the MOOC on implementation research for reaching out to eld researchers and public health practitioners who are facing problems in the implementation of control programmes in low-and middle-income countries. Contributions To The Literature Implementation research can make a critical difference to health outcomes, particularly in low-and middle-income countries and in connection with infectious diseases of poverty. There is under-utilisation of implementation research in low-and middle-income countries due to limited knowledge and training. Free online training, such as that offered in a MOOC, provides a way to increase knowledge of implementation research to eld researchers and public health practitioners facing problems in the implementation of control programmes in low-and middle-income countries.

BackgroundHealth research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such issue: harmonising the evaluation requirements of different funders.MethodsWe identified and analysed health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012, using a mixed methods approach involving structured qualitative analyses of documents, a stakeholder survey and consultations with key contacts in health RCS funding agencies.ResultsThe frameworks were intended for use predominantly by the organisations themselves, and most were oriented primarily towards funders’ internal organisational performance requirements. The frameworks made limited reference to theories that specifically concern RCS. Generic devices, such as logical frameworks, were typically used to document activities, outputs and outcomes, but with little emphasis on exploring underlying assumptions or contextual constraints. Usage of the ESSENCE framework appeared limited.ConclusionsWe believe that there is scope for improving frameworks through the incorporation of more accessible information about how to do evaluation in practice; greater involvement of stakeholders, following evaluation capacity building principles; greater emphasis on explaining underlying rationales of frameworks; and structuring frameworks so that they separate generic and project-specific aspects of health RCS evaluation. The third and fourth of these improvements might assist harmonisation.

Strengthening research capacity in low- and middle-income countries is one of the most effective ways of advancing their health and development but the complexity and heterogeneity of health research capacity strengthening (RCS) initiatives means it is difficult to evaluate their effectiveness. Our study aimed to enhance understanding about these difficulties and to make recommendations about how to make health RCS evaluations more effective. Through discussions and surveys of health RCS funders, including the ESSENCE on Health Research initiative, we identified themes that were important to health RCS funders and used these to guide a systematic analysis of their evaluation reports. Eighteen reports, produced between 2000 and 2013, representing 12 evaluations, were purposefully selected from 54 reports provided by the funders to provide maximum variety. Text from the reports was extracted independently by two authors against a pre-designed framework. Information about the health RCS approaches, tensions and suggested solutions was re-constructed into a narrative. Throughout the process contacts in the health RCS funder agencies were involved in helping us to validate and interpret our results. The focus of the health RCS evaluations ranged from individuals and institutions to national, regional and global levels. Our analysis identified tensions around how much stakeholders should participate in an evaluation, the appropriate balance between measuring and learning and between a focus on short-term processes vs longer-term impact and sustainability. Suggested solutions to these tensions included early and ongoing stakeholder engagement in planning and evaluating health RCS, modelling of impact pathways and rapid assimilation of lessons learned for continuous improvement of decision making and programming. The use of developmental approaches could improve health RCS evaluations by addressing common tensions and promoting sustainability. Sharing learning about how to do robust and useful health RCS evaluations should happen alongside, not after, health RCS efforts.

BackgroundThe monitoring and evaluation of health research capacity strengthening (health RCS) commonly involves documenting activities and outputs using indicators or metrics. We sought to catalogue the types of indicators being used to evaluate health RCS and to assess potential gaps in quality and coverage.MethodsWe purposively selected twelve evaluations to maximize diversity in health RCS, funders, countries, and approaches to evaluation. We explored the quality of the indicators and extracted them into a matrix across individual, institutional, and national/regional/network levels, based on a matrix in the ESSENCE Planning, Monitoring and Evaluation framework. We synthesized across potential impact pathways (activities to outputs to outcomes) and iteratively checked our findings with key health RCS evaluation stakeholders.ResultsEvaluations varied remarkably in the strengths of their evaluation designs. The validity of indicators and potential biases were documented in a minority of reports. Indicators were primarily of activities, outputs, or outcomes, with little on their inter-relationships. Individual level indicators tended to be more quantitative, comparable, and attentive to equity considerations. Institutional and national–international level indicators were extremely diverse. Although linkage of activities through outputs to outcomes within evaluations was limited, across the evaluations we were able to construct potential pathways of change and assemble corresponding indicators.ConclusionsOpportunities for improving health RCS evaluations include work on indicator measurement properties and development of indicators which better encompass relationships with knowledge users. Greater attention to evaluation design, prospective indicator measurement, and systematic linkage of indicators in keeping with theories of change could provide more robust evidence on outcomes of health RCS.