Performance model’s development: a novel approach encompassing ontology-based data access and visual analytics

Abstract

The quantitative evaluation of research is currently carried out by means of indicators calculated on data extracted and integrated by analysts who elaborate them by creating illustrative tables and plots of results. In this approach, the robustness of the metrics used and the possibility for users of the metrics to intervene in the evaluation process are completely neglected. We propose a new approach which is able to move forward, from indicators’ development to an interactive performance model’s development. It combines the advantages of the ontology-based data access paradigm with the flexibility and robustness of a visual analytics environment putting the consumer/stakeholder at the centre of the evaluation. A detailed description of such an approach is presented in the paper. The approach is illustrated and evaluated trough a comprehensive user’s study that proves the added capabilities and the benefits that a user of performance models can have by using this approach.

Accounting for quality in data integration systems: a completeness-aware integration approach

“…Lenzerini and Daraio (2019) discuss the main challenges, approaches and solutions available for integrating data on research, higher education and innovation, consolidating existing research on the topic, including Daraio et al (2016a) which introduce Sapientia, the ontology of multidimensional assessment of research and Daraio et al (2016b) that highlighted and discussed the main advantages on an OBDM approach residing in the openness, interoperability and data quality. Recently, Angelini et al (2020) showed the usefulness of Sapientia and OBDM combined with visual analytics to develop general models of performance indicators.…”

A visual analytics approach for the assessment of information quality of performance models—a software review

“…This is especially important given the heterogeneity of data sources, the different formats that can still convey similar semantics, and the importance that feature selection can have on the definition of a performance model. Angelini et al (2020) provide a workflow for dynamic creation and assessment of a performance model for evaluating research activities. This workflow is based on ontological modeling of the data sources, instantiated in the Sapientia ontology, that align semantically the contents coming from different sources (e.g., ETER, Scopus, Wos, and so on).…”

“…Daraio et al (2016) show the benefits of an ontology-based data integration approach for data quality in an open environment. Angelini et al (2020) present the advantages of Visual Analytics for the development of performance models. In this paper we take a step further and present a visual analytic environment featured to performance model's development which includes data quality procedures and tests.…”