Beyond the metrics of health research performance in African countries

Abstract

While it is important to be able to evaluate and measure a country’s performance in health research (HR), HR systems are complex and multifaceted in nature. As such, attempts at measurement can suffer several limitations which risk leading to inadequate indices or representations. In this study, we critically review common indicators of HR capacity and performance and explore their strengths and limitations. The paper is informed by review of data sources and documents, combined with interviews and peer-to-peer learning activities conducted with officials working in health and education ministries in a set of nine African countries. We find that many metrics that can assess HR performance have gaps in the conceptualisation or fail to address local contextual realities, which makes it a challenge to interpret them in relation to other theoretical constructs. Our study identified several concepts that are excluded from current definitions of indicators and systems of metrics for HR performance. These omissions may be particularly important for interpreting HR performance within the context and processes of HR in African countries, and thus challenging the relevance, utility, appropriateness and acceptability of universal measures of HR in the region. We discuss the challenges that scholars may find in conceptualising such a complex phenomenon—including the different and competing viewpoints of stakeholders, in setting objectives of HR measurement work, and in navigating the realities of empirical measurement where missing or partial data may necessitate that proxies or alternative indicators may be chosen. These findings are important to ensure that the global health community does not rely on over-simplistic evaluations of HR when analysing and planning for improvements in low-income and middle-income countries.

Measuring health science research and development in Africa: mapping the available data

“…However, there is support for the indicators we selected for our framework from the literature discussing measurement of health R&D globally and in Africa [ 35 , 107 ]. Nevertheless, we acknowledge that there are multiple challenges and issues with measuring HSciR performance using universal indicators in the contexts of African national health research systems [ 108 ].…”

Mapping regional cooperation of state actors for health research systems in Africa: A social network analysis

“…However, without significant correlations between increased regional connections and improved national health research performance, the benefits of African regionalism for health research are inconclusive and merit further exploration. For instance, a critical perspective on the selection and use of global metrics to measure performance of health research systems in Africa is needed, which we have reflected upon elsewhere [61]. The HSciR performance metrics which have been developed, defined, collected, and disseminated for the most part by institutions in the global north have important consequences when they are recommended or imposed as indicators for use in decision-making and monitoring health research systems in other contexts; there are several research and collaboration processes related to power, equity, and ownership hidden behind and within these numbers [61].…”

“…For instance, a critical perspective on the selection and use of global metrics to measure performance of health research systems in Africa is needed, which we have reflected upon elsewhere [61]. The HSciR performance metrics which have been developed, defined, collected, and disseminated for the most part by institutions in the global north have important consequences when they are recommended or imposed as indicators for use in decision-making and monitoring health research systems in other contexts; there are several research and collaboration processes related to power, equity, and ownership hidden behind and within these numbers [61]. Despite methodological, ethical, political and practical issues with HSciR performance evaluation and use of these metrics for African institutions [61][62][63], these are among the core set of indicators used to track, benchmark, and analyse HSciR development and performance at a global level in the [64][65][66].…”

“…The HSciR performance metrics which have been developed, defined, collected, and disseminated for the most part by institutions in the global north have important consequences when they are recommended or imposed as indicators for use in decision-making and monitoring health research systems in other contexts; there are several research and collaboration processes related to power, equity, and ownership hidden behind and within these numbers [61]. Despite methodological, ethical, political and practical issues with HSciR performance evaluation and use of these metrics for African institutions [61][62][63], these are among the core set of indicators used to track, benchmark, and analyse HSciR development and performance at a global level in the [64][65][66]. In our work mapping available indicators for HSciR in all 54 African countries, we found a nuanced picture, whereby health research systems performed differently across indicators using the state jurisdiction as the unit of analysis (national level) [44], a finding which has also been shown in studies using indicators of knowledge economies in Africa [67].…”

“…Aggregating health research indicators to the regional level is one possibility, but this would reproduce issues with data availability and appropriateness. Indeed, the relevance and use of standard international metrics for evaluating health research performance in African countries is not without critique and limitations as well [61]. Another option would be to review health research policy diffusion or research uptake in a regional network as a measure of cooperation effects.…”