Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators

Abstract

While bibliometric indicators, such as the journal impact factor, have long played an important role in many STEM disciplines it has been repeatedly shown that established bibliometric methods have limited use in the humanities. Using a questionnaire on metrics use and publication practices in Australia and Sweden, we tested the assumption that indicators play a minor role among humanities scholars. Our findings show that our respondents use indicators to a considerable degree, with a range of indicators and rankings being employed. The scholars use metrics as part of institutional policy, in CVs and applications, as well as for general promotion of their work. Notable in our results is that a much larger share of researchers (62%) in Australia used metrics compared to Sweden (14%). Scholar's attitudes regarding bibliometrics are mixed; many are critical of these measures, while at the same time feeling pressured to use them. One main tension described by our respondents is between intradisciplinary criteria of quality and formalized indicators, and negotiating these “orders of worth” is a challenging balancing act, especially for younger researchers.

“When You Use Social Media You Are Not Working”: Barriers for the Use of Metrics in Social Sciences

“…While identifying social scientists' concerns regarding researchers' professional usage of social media might reveal what would need to be done to increase Social Science publications' coverage on social media, altmetrics' (and usage metrics') usefulness for the discipline is limited by at least one other major factor: their acceptance among stakeholders-most of which will be researchers. Hammarfelt and Haddow (2018) analyzed the attitudes of Australian and Swedish researchers from the Social Sciences and Humanities toward bibliometric indicators, finding that "scholar's attitudes regarding bibliometrics are mixed; many are critical of these measures, while at the same time feeling pressured to use them." Also they found the shares of researchers that had already used bibliometrics to vary significantly between the two countries.…”

Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined

“…Section 2.2 suggested the use of alternative nested H indexes based on linear and cubic fittings of standardized numbers of articles and citations, whereas Section 2.3 presented two-dimensional graphs based on alternative nested H and G indexes. These approaches would reduce incentives to engage in tactical or opportunistic behaviors in publication and citation by authors and journal editors [76][77][78][79][80][81][82], and should reduce discrimination against heterodox and interdisciplinary PHs that would be characterized by few citations and few articles [83,84]. Table 4 summarizes suggested warning symptoms that could be used to identify potentially questionable practices by editors and authors, although future experimental work based on analytical insights will be necessary to test whether these symptoms truly indicate manipulation of the PH quality.…”

Taking Comfort in Points: The Appeal of the Norwegian Model in Sweden

“…One of the first studies to look at the how the model is used in the humanities was Hammarfelt and de Rijcke (2015), and the implementation of the model in a faculty of social sciences was studied by Edlund and Wedlin (2017). Here I will present a few insights from a recent survey of humanities scholars and social scientists in Sweden and Australia (Hammarfelt & Haddow, 2018;Haddow & Hammarfelt, to appear.). Short free text answers provided by Swedish respondents to this survey-which was about metric use and publication…”

“…When researchers express their views, it is often the negative consequences of assessment procedures that are in focus. However, changing publication practices, which by many is associated with the introduction of the Norwegian system, is welcomed by some researchers (Hammarfelt & Haddow, 2018). Thus, it is not uncommon that the use of the Norwegian list associated with a more general…”

“…Moreover, it is likely that the "publication indicator" will have considerable influence in various contexts even if it is formerly abandoned in a performance-based allocation system. An example of this phenomenon, what we may call the "after-life of indicators", is the ERA-list in Australia which was formally used only for a short period, yet this ranked list of journals still plays an important role among researchers (Hammarfelt & Haddow, 2018). Previous studies has pointed to the problem of the "Norwegian publication indicator" being used in unintended ways (Aagaard, 2015;Hammarfelt et al, 2016), and actions, such as "inter-instutional learning areas" has been implemented in order to limit inappropriate use (MLE on Performance-based Funding of Public Research Organisations 2018.)…”