2017
DOI: 10.1515/jdis-2017-0018
|View full text |Cite
|
Sign up to set email alerts
|

A Framework for the Assessment of Research and Its Impacts

Abstract: This paper proposes a holistic framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions, including in an original way, data as a main dimension, together with theory and methodology. Each dimension of the framework is further characterized by three main building blocks: education, research, and innovation (theory); efficiency, effectiveness, and impact (methodology); and availability, interoperability, and “unit-free” property (data)… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1
1

Citation Types

0
14
0

Year Published

2019
2019
2022
2022

Publication Types

Select...
6
1

Relationship

5
2

Authors

Journals

citations
Cited by 9 publications
(14 citation statements)
references
References 132 publications
0
14
0
Order By: Relevance
“…The development of models of indicators or metrics for a quantitative assessment requires a comprehensive framework, which should include the specification of the underlying theory, methodology, and data properties. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics (Daraio, 2017). Daraio and Glanzel (2016) identified the following critical issues: i) data quality issues (OECD, 2011) including completeness, validity, accuracy, consistency, availability, and timeliness; ii) comparability problems related to heterogeneous definitions of the variables, data collection practices, and databases; iii) lack of standardization; iv) lack of interoperability; v) lack of modularization; vi) problems of classification; vii) difficulties in the creation of concordance tables among different classification schemes; viii) problems and costs of the extensibility of the system; ix) problems and costs of updating of the system.…”
Section: History Of Eter and Its Political Importance For Research Onmentioning
confidence: 99%
“…The development of models of indicators or metrics for a quantitative assessment requires a comprehensive framework, which should include the specification of the underlying theory, methodology, and data properties. Models of metrics are necessary to assess the meaning, validity, and robustness of metrics (Daraio, 2017). Daraio and Glanzel (2016) identified the following critical issues: i) data quality issues (OECD, 2011) including completeness, validity, accuracy, consistency, availability, and timeliness; ii) comparability problems related to heterogeneous definitions of the variables, data collection practices, and databases; iii) lack of standardization; iv) lack of interoperability; v) lack of modularization; vi) problems of classification; vii) difficulties in the creation of concordance tables among different classification schemes; viii) problems and costs of the extensibility of the system; ix) problems and costs of updating of the system.…”
Section: History Of Eter and Its Political Importance For Research Onmentioning
confidence: 99%
“…The main components of a performance evaluation model can be found in Daraio (2017Daraio ( , 2019. We have actors that are involved in processes which consist in the combination and or transformation of inputs in outputs, taking into account the main objectives of the activities.…”
Section: Moving From Indicators Development To Performance Models Devmentioning
confidence: 99%
“…These changes produce different effects (see further details and references in Daraio 2019, Table 24.2, p. 644) (i) on the demand side (those that ask for research assessment) including an increase of institutional and internal assessments, (ii) on the supply side (those that offer research assessment) including proliferation of rankings, development of Altmetrics, open access repositories, new assessment tools and desktop bibliometrics), (iii) on scholars (the increase of "publish or perish" pressure, impact on the incentives, behaviour and misconduct, and increasing critics against traditional bibliometric indicators), (iv) on the assessment process (increasing the complexity of the research assessment) and on the indicators' development. Daraio (2017) showed that the formulation of models (in this paper we will use metrics and indicators interchangeably) is necessary to assess the meaning, validity and robustness of metrics. It was observed that developing models is important for learning about the explicit consequences of assumptions, for testing the assumptions, for documenting and verifying the assumptions, for systematizing the problem and the choices done.…”
Section: Introductionmentioning
confidence: 99%
See 1 more Smart Citation
“…Daraio and Bonaccorsi () illustrate the design of a possible information system to integrate microdata on universities with other sources with the aim of overcoming rankings by linking data in an open platform. In particular, the new developments may be useful to consider data quality dimensions and the openness of the data platform, although the limits of data and their availability and interoperability still remain a critical issue (Borgman, ; Daraio, Lenzerini, Leporelli, Naggar, et al, ) that need to be considered in the development of metrics for the assessment of research, education and innovation (Daraio, ).…”
Section: Research and Higher Education In Today's Societymentioning
confidence: 99%