Stop evaluating science: A historical-sociological argument

Hallonsten, Olof

doi:10.1177/0539018421992204

Cited by 54 publications

(59 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While numbers of citations to an article likely, on average, correlates with the quality of the article and its findings, there are many reasons for citing articles besides appreciation of their quality, and the variation in practices of citation across the sciences makes any uniform measure problematic, to say the least (Muller, 2018: 79). But it is easy to understand the lure: The simplicity and straightforwardness of citation counts and aggregate measures make them readily accessible to non-expert policymakers and bureaucrats, and also, to some extent, to the broader general public, none of whom are equipped to grasp the results of, let alone undertake, the only viable method for assessing the deeper quality and relevance of scientific research, namely expert evaluation on a case-to-case basis (Hallonsten, 2021).…”

Section: Peer Review and Organized Skepticismmentioning

confidence: 99%

On the essential role of organized skepticism in science’s “internal and lawful autonomy” (Eigengesetzlichkeit)

Hallonsten¹

2021

Journal of Classical Sociology

Self Cite

View full text Add to dashboard Cite

The institution of science is said to be under pressure from political, economic and social interests, manifested in alleged bureaucratization, managerial reforms, anti-intellectual movements on university campuses, and widespread questioning of expert knowledge in society. Commercialization of academic publishing and the growth of competitive funding have increased the importance of journal and grant peer review in science and seem also to have contributed to the proliferation of false impressions about the nature of scientific knowledge production and science’s role in society. In this article, these developments are problematized and put into perspective with the help of classic sociological theory in a Weberian and Mertonian tradition, in an attempt to shed new light on the debate on the governance and institutional autonomy of science. First, academic science is identified as a Weberian value sphere with “internal and lawful autonomy” ( Eigengesetzlichkeit), and the broader functionalist context of this supposition is discussed. Second, Merton’s theory of the normative structure of science is used to give specific content to Eigengesetzlichkeit in the case of science. Third, the concept of organized skepticism is developed to represent a range of social patterns ubiquitous in scientific practice, and its epistemological and sociological foundations are discussed. Organized skepticism is thus identified as the essential feature of science’s Eigengesetzlichkeit.

show abstract

Section: Peer Review and Organized Skepticismmentioning

confidence: 99%

On the essential role of organized skepticism in science’s “internal and lawful autonomy” (Eigengesetzlichkeit)

Hallonsten¹

2021

Journal of Classical Sociology

Self Cite

View full text Add to dashboard Cite

show abstract

“…All of these factors converge towards essentially one issue: should science be further rationalized and if so, to what extent and how? Hallonsten (2021) criticizes what appears to be the growing consensus, or perhaps implicit consent, in society that science should be further rationalized through exogenous interference.…”

Section: Mots-clésmentioning

confidence: 99%

“…Despite the provocative tone, Hallonsten (2021) ultimately supports a middle of the road approach, as he repeats Linda Butler's plea for sanity. As quantitative performance measures were being introduced to Australia and the United Kingdom, Butler (2007) promoted a balanced approach, arguing that although metrics had their place, for instance by making the process more efficient and cost effective, qualitative peer review should remain the keystone in research evaluation.…”

Section: Mots-clésmentioning

confidence: 99%

“…The perverse effects of the current benchmarks Hallonsten (2021) argues that much of the current practice in quantitative performance evaluation is pointless and counterproductive. While the former predicate adjective is needlessly provocative, there is already a growing literature criticizing the unbalanced usage of metrics in the evaluation of science (Gendron, 2008;Adler and Harzing, 2009;Sauder, 2009, 2016).…”

Section: Mots-clésmentioning

confidence: 99%

See 1 more Smart Citation

If you do not deign to quantify, someone else will do it for you: In support of a balanced approach to the evaluation of science

Lizotte

2021

Social Science Information

View full text Add to dashboard Cite

This is a commentary in support of Olof Hallonsten’s historical-sociological argument for countering the growing distrust and governance of science. From this starting point, the problem of quantification in the evaluation of science is addressed and several examples of the unintended consequences of the currently available metrics are discussed. In particular, the issue of quantification is discussed in regard to the modality of scientific research, power and research and the peer relationship. Although in approval with Hallonsten’s argument for reversing the burden of proof, reasonable skepticism is expressed regarding the persuasiveness that this counter-rhetoric will have on members of parliament, public servants and university administrators. If this long-term goal is to be accomplished, it is argued that concrete actions must be pursued in the short and medium term. In this spirit, several suggestions are formulated to further this agenda, most notably greater support for intellectual diversity, greater participation and readership in science studies by science practitioners and the promotion of the comparative approach for understanding the different ways that metrics are actually used in practice. Finally, I argue that the refusal of participating in the quantification of science is bound to hinder applied critical thinking and will most likely and regrettably exacerbate its current perverse effects.

show abstract

“…The value of science is increasingly measured by numbers, from simple publication counts to more complex performance metrics. Olof Hallonsten (2021) encourages us to reject the external evaluation of science which relies on the 'means of quantitative metrics' to measure science for the primary purpose of 'economic growth'. Using the 'authority of numbers' to assert public control over professional practices is not limited to the domain of science, but is a general postwar sociological phenomenon seen in many sectors such as education, medicine, social work and accounting (Desrosières, 1998;Espeland and Stevens, 2008;Porter, 1995;Power, 1997).…”

Section: Science Needs More External Evaluation Not Lessmentioning

confidence: 99%

Science needs more external evaluation, not less

Knaapen

2021

Social Science Information

View full text Add to dashboard Cite

When science is evaluated by bureaucrats and administrators, it is usually done by quantified performance metrics, for the purpose of economic productivity. Olof Hallonsten criticizes both the means (quantification) and purpose (economization) of such external evaluation. I share the concern that such neoliberal performance metrics are shallow, over-simplified and inaccurate, but differ in how best to oppose this reductionism. Hallonsten proposes to replace quantitative performance metrics with qualitative in-depth evaluation of science, which would keep evaluation internal to scientific communities. I argue that such qualitative internal evaluation will not be enough to challenge current external evaluation since it does little to counteract neoliberal politics, and fails to provide the accountability that science owes the public. To assure that the many worthy purposes of science (i.e. truth, democracy, well-being, justice) are valued and pursued, I argue science needs more and more diverse external evaluation. The diversification of science evaluation can go in many directions: towards both quantified performance metrics and qualitative internal assessments and beyond economic productivity to value science’s broader societal contributions. In addition to administrators and public servants, science evaluators must also include diverse counterpublics of scientists: civil society, journalists, interested lay public and scientists themselves. More diverse external evaluation is perhaps no more accurate than neoliberal quantified metrics, but by valuing the myriad contributions of science and the diversity of its producers and users, it is hopefully less partial and perhaps more just.

show abstract

Stop evaluating science: A historical-sociological argument

Cited by 54 publications

References 57 publications

On the essential role of organized skepticism in science’s “internal and lawful autonomy” (Eigengesetzlichkeit)

On the essential role of organized skepticism in science’s “internal and lawful autonomy” (Eigengesetzlichkeit)

If you do not deign to quantify, someone else will do it for you: In support of a balanced approach to the evaluation of science

Science needs more external evaluation, not less

Contact Info

Product

Resources

About