Measuring University Performance by Means of Composite Indicators: A Robustness Analysis of the Composite Measure Used for the Benchmark of Italian Universities

Abstract

Composite indicators (CIs) are a useful communication tool for conveying summary performance information in a relatively simple way. They are used widely in various sectors in public services. In Italy, they are also employed to allocate a quota of public funding to the universities. Despite increasing use, CIs value and benefit are still under debate and their suitability for the benchmark of universities is challenged by methodological issues and the multi-dimensional facet of higher education institutions. The issue addressed in the present work is whether the composite measure employed by the Italian Government is an accurate tool for evaluating, ranking and funding universities. For this purpose-after a revision of the indicators, weighting and aggregation methods put forward at the national level-CIs have been computed for the overall university performance, employing the indicators chosen at the national level and combining various weighting and normalisation schemes. The variation in the rankings assigned by the CIs to the given universities has been then explored to gauge its robustness. In the light of the results, some technical and analytical issues involved in the design of CIs and their policy implications are also discussed.

Cross-country analysis of higher education institutions’ efficiency: The role of strategic positioning

“…In such a context, the problem of measuring the performance of institutions which deliberately decide to differentiate from others by means of different missions is relevant (see also section 2.2). A possible approach is to try aggregating the performance dimensions into a single composite indicator (as in Gnaldi and Ranalli 2016). An alternative way is to employ methods that allow assessing multiple outputs and weighting them differently according to the strategic profileand this is the approach taken in this paper.…”

Dynamic quality assessment as a criterion for the sustainable development of social services in cross-border regions

“…Given the complexity and diversity of factors that determine the quality of social services perceived by consumers, it is important not only to calculate comprehensive indicators of satisfaction, but also to identify successful practices in managing the development of individual industries [2]. The ranking of territories according to the values of consumer assessments of the quality of services received in different years has a definite impact on the image and attractiveness of regions, including for investment in new projects [3]. However, to create the conditions for quality improvement, an in-depth analysis of the sustainability of development of individual service industries is needed based on the assessment of customer satisfaction over time [4].…”

Benefit of the doubt approach to assessing the research performance of Australian universities

“…In addition to decision makers in Higher Education systems, rankings are of interest to students selecting a university and to employers who consider the rankings when recruiting employees, and other stakeholders use these rankings to make decisions related to, for example, funding or sponsorship. Despite the criticism of rankings based on CIs, they are likely the only widely available benchmark for Higher Education institutions (Gnaldi & Ranalli, ; Marconi & Ritzen, ).…”