Using normative ethics for building a good evaluation of research practices: towards the assessment of researcher’s virtues

Abstract

In this paper, we propose the adoption of moral philosophy and in particular normative ethics, to clarify the concept of “good” evaluation of “research practices”. Using MacIntyre (1985)’s notion of a practice we argue that research is a form of social practice. As a result of this characterization, we claim that research practice typically requires three typologies of researcher: the leader, the good researcher and the honest researcher. Reflecting on what is a “good” research practice and on what is the role of researchers in it provides insight into some aspects of both the self-assessment process and how this promotes individual improvement. Moreover, this kind of reflection helps us to describe the functions (missions) of the research practices. A “good” evaluation should take into account all the building constituents of a “good” research practice and should be able to discriminate between good and bad research practices, while enforcing the functions of good research practices. We believe that these reflections may be the starting point for a paradigm shift in the evaluation of research practices which replaces an evaluation centred on products with an evaluation focused on the functions of these practices. In the last sections of the paper, we introduce and discuss an important aspect for the implementation of the proposed framework. This relates to the assessment of the virtues of researchers involved in a good research practice. Some examples of questions and preliminary items to include in a questionnaire for the assessment of Virtues in Research Practices are also provided.

In Defense of Merit to Overcome Merit

“…This consideration leads us to think that in order to being able to unambiguously interpret and use bibliometric indicators in research evaluation we have to consider also individual and social characteristics of researchers. Building on the notion of practice of MacIntyre (1985) , in Daraio and Vaccari (2020) we argue that the most appropriate level of analysis for building a “good evaluation” is that of “research practice,” intended as a form of social practice. The adoption of this level of analysis requires a paradigm shift in the assessment of research from an evaluation centered only on products (outputs of the research, i.e.…”

El papel de la investigación en la educación jurídica: Un problema de poder y colonialidad

“…El uso de normas éticas para construir buenas prácticas en la investigación que, a la vez, puedan ser usadas como referentes para la evaluación de la indagación científica, no ha sido considerado (Daraio y Vaccari, 2020). Solo parecen interesar los productos en términos cuantitativos.…”

How should evaluation be? Is a good evaluation of research also just? Towards the implementation of good evaluation

“…Although there is a proliferation of increasingly sophisticated quantitative methods for evaluating research, there is still a lack of clarity on how to understand and operationalize the notion of "good" evaluation of research practices. In a previous recent work (Daraio & Vaccari, 2020) we recognize researchers' motivations and their specific traits of character (i.e. researchers' virtues) as important factors that must be considered to make a "good" evaluation.…”

“…Virtues allow the goods of the practice to be achieved and are appropriate to the different roles that researchers play according to their skills and experiences. In a good research practice, in fact, there are three main kinds of researchers involved: leaders, good researchers and honest researchers (Daraio & Vaccari, 2020). The main result of this recent study is the preliminary version of a questionnaire to assess the researcher's virtues, to be able to complement current research evaluations with additional elements referring to a scholar's motivations and weak elements in her character currently excluded by bibliometric indicators in use.…”

“…In this paper we maintain that evaluation must be good and investigate the connection between a good evaluation and a just evaluation. We show that adopting the notion of good evaluation proposed in Daraio and Vaccari (2020) automatically leads us to a just evaluation that encompasses the fairness but also goes beyond it by also relying on a broader general concept of justice. Moreover, we propose the development of ontologies and taxonomic knowledge as a way of pursuing the implementation of the good evaluation through a questionnaire on researchers' virtues.…”