How peer-review constrains cognition: on the frontline in the knowledge sector

Abstract

Peer-review is neither reliable, fair, nor a valid basis for predicting ‘impact’: as quality control, peer-review is not fit for purpose. Endorsing the consensus, I offer a reframing: while a normative social process, peer-review also shapes the writing of a scientific paper. In so far as ‘cognition’ describes enabling conditions for flexible behavior, the practices of peer-review thus constrain knowledge-making. To pursue cognitive functions of peer-review, however, manuscripts must be seen as ‘symbolizations’, replicable patterns that use technologically enabled activity. On this bio-cognitive view, peer-review constrains knowledge-making by writers, editors, reviewers. Authors are prompted to recursively re-aggregate symbolizations to present what are deemed acceptable knowledge claims. How, then, can recursive re-embodiment be explored? In illustration, I sketch how the paper’s own content came to be re-aggregated: agonistic review drove reformatting of argument structure, changes in rhetorical ploys and careful choice of wordings. For this reason, the paper’s knowledge-claims can be traced to human activity that occurs in distributed cognitive systems. Peer-review is on the frontline in the knowledge sector in that it delimits what can count as knowing. Its systemic nature is therefore crucial to not only discipline-centered ‘real’ science but also its ‘post-academic’ counterparts.

Fragments of peer review: A quantitative analysis of the literature (1969-2015)

“…The primacy of the link between peer review and the concept of “quality” was confirmed by looking at nouns/verbs/adjectives in the paper titles (“quality” appearing 527 times against “journal” appearing 454 times or “research”, 434 times) and in the abstracts (“quality” recurring 2208 times against “research” 2038 or medical 1014 times). This would confirm that peer review has been mainly viewed as a “quality control” process rather than a collaborative process that would aim at increasing the knowledge value of a manuscript [ 17 ].…”

Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation

“…We relate these measures to the citedness of the published papers. Our work builds on previous observations about (a) the role of actors involved in the research and publication process which has argued that peer review is inherent in the research process (Rigby and Edler 2005 ); (b) work which emphasises the contribution which reviewers themselves make to the content and therefore citedness of papers, particularly in the social and behavioural sciences; and (c) the claims by Cowley ( 2015 ) of peer review as systemic, distributed, complex knowledge production process within which larger frame of reference journal peer review occurs. In respect of these observations, and in particular our focus on the role of reviewers, we note the words of Bakanic et al ( 1987 , p. 641) who claimed that ‘expert peer participation may not come until after the journal’s review is underway… and where manuscript review, revision, and resubmission process are vital contributions to the construction of the scholarship reported’—a claim made fully a decade and a half before Frey ( 2003 ) asked authors to choose between writing their own work and academic success.…”

Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation

“…Measuring quality through journal impact factors and/or article citations is problematic, due to the disproportionate effect of a few highly cited articles (e.g., Baum 2011 ). Previous research also suggests that the quality of peer review is a complex concept, which strongly depends on the multiple, often ambiguous functions that this process has (e.g., Cowley 2015 ; Lamont et al 2009 ; Ma et al 2013 ; Pontille and Torny 2015 ; Ragone et al 2013 ). For instance, current research on peer review sees it as an engine to select excellent, innovative or rigorous research and avoid publishing below standard contributions (Nedić and Dekanski 2016 ).…”