Publication Anxiety, Quality, and Journal Rankings: Researcher Views

Abstract

Journal ranking systems are increasingly used to measure research performance of academics and universities. A growing number of academic commentators have voiced concerns of possible undesirable outcomes such as increased publication anxiety and an increase in safe and conforming research, but there have been few empirical studies on the possible effects. To address this gap, we surveyed Information Systems (IS) academics who published in one of three key IS conferences in 2013, to gather their views of the effects of journal ranking systems. Overall, we found that the concerns in the literature were strongly reflected in the views of those surveyed. Academics believe the system has greatly increased their publication anxiety. While most believed that the quality of published research had improved, researchers believe the ranking systems inhibit innovative, risky research, and encourages safe, conforming, mainstream research.

Dialectic Antidotes to Critics of the Technology Acceptance Model: Conceptual, Methodological, and Replication Treatments for Behavioural Modelling in Technology-Mediated Environments

“…For behavioural modelling in technology-mediated environments, ground-breaking contributions are often evaluated more favourably than incremental contributions delivered by replications (Coulthard and Keller, 2016;Lim, 2016). Direct replication of previously published empirical studies is challenging because of likely differences in study location, personnel, and time, as well as the threat of evolving socioeconomic changes and rapid technological proliferation, and thus the general understanding of replication herein encapsulates the action or process of repeating the test of similar propositions by using similar and dissimilar methodological procedures for behavioural modelling (e.g.…”

Special Issue: Professions and Professionalism in Market-Driven Societies

“…Indeed, some authors argue that such quantified criteria of assessment result in the over-valorization of quantity over quality (Kallio & Kallio, 2014;Olssen & Peters, 2005;Stelmach & Wollf, 2011), to the extent that evaluation has become constrained by standardized scores based upon rankings and ratings (Lane, 2010). These may encourage conformism since publication norms tend to promote standard work rather than critical innovation (Coulthard & Keller, 2016;Sousa, 2011), potentially jeopardizing objectivity and integrity in publication choices (Fanelli, 2010) as well as individualism in research (Sousa, 2011). On the other hand, dependence upon sources of self-financing may constrain academic organizations' decisions regarding their teaching and research to cater to the educational marketplace, leading them to approximate educational demand (Van Dalen & Henkens, 2012).…”

“…Survival in an increasingly competitive and uncertain market thereby transforms ratings and rankings into the instruments through which HEIs measure merit and are themselves evaluated for merit (Stelmach & Wolff, 2011). Moreover, marketization may risk the independence of academic and scientific research and publication since projects, and the interpretation of results can be influenced by the priorities of private interests (Coulthard & Keller, 2016). These dynamics are a strong pressure on teachers and researchers and have a strong effect upon their motivations (Anderson, Johnson, & Saha, 2002).…”

Resource-Based View of Information Systems: Sustainable and Transient Competitive Advantage Perspectives

“…The keys to sustaining the competitive advantage of information systems can come from greater alignment between alternative (e.g. business) and information system strategies (Choe, 2003;Liang & You, 2009), complementary synergies with other internal and external firm resources (Hulland et al, 2007;Ravichandran & Lertwongsatien, 2005), innovation (Grant, 1999;Shin, 2006), which may be incremental, or even better, ground-breaking (Coulthard & Keller, 2016;Lim, 2016), and systematic and timely upgrades of organizational (e.g. expertise, skills) and functional capabilities (e.g.…”