Peer Review Personas

Belojevic, Nina; Sayers, Jentery

doi:10.3998/3336451.0017.304

“…However, we recognize that any technological advance is rarely innocent or unbiased, and while Web 2.0 technologies open up the possibility for increased participation in peer review, it would still not be inherently democratic ( Elkhatib et al , 2015 ). As Belojevic et al (2014) remark, when considering tying reputation engines to peer review, we must be aware that this comes with implications for values, norms, privilege and bias, and the industrialization of the process ( Lee et al , 2013 ). Peer review is socially and culturally embedded in scholarly communities and has an inherent diversity in values and processes, which we must have a deep awareness of and appreciation for.…”

Section: Discussionmentioning

confidence: 99%

“…The points directly reflect an individual’s contribution to that specific research community. Such processes ultimately have a very low barrier to entry, but also expose peer review to potential gamification through integration with a reputation engine, a social bias which proliferates through any technoculture ( Belojevic et al , 2014 ).…”

Section: Potential Future Modelsmentioning

confidence: 99%

A multi-disciplinary perspective on emergent and future innovations in peer review

Tennant

¹

,

Dugan

²

,

Graziotin

³

et al. 2017

View full text Add to dashboard Cite

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

show abstract

“…As Belojevic et al (2014) remark, when considering tying reputation engines to peer review, we must be aware that this comes with implications for values, norms, privilege and bias, and the industrialization of the process ( Lee et al , 2013). Peer review is socially and culturally embedded in scholarly communities and has an inherent diversity in values and processes, which we must have a deep awareness of and appreciation for.…”

Section: Discussionmentioning

confidence: 99%

A multi-disciplinary perspective on emergent and future innovations in peer review

Dugan

¹

,

Graziotin

²

,

Jacques

³

et al. 2017

View full text Add to dashboard Cite

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure? Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research. With the advent of web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review. We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation. Ideally, any new systems will demonstrate that they out-perform and reduce the biases of existing models as much as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages. We also propose a novel hybrid platform model that could, at least partially, resolve many of the socio-technical issues associated with peer review, and potentially disrupt the entire scholarly communication system. Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

show abstract

“…The second highest links (85) were from a direct URL link to the digital object identier. The the third and the last were from library.wiley.com (3).…”

Section: Experiments Setup and Methodologymentioning

confidence: 99%

“…This community has the signicant fraction of the participants having deep expertise in the domain area. Any scholarly articles linked in Stack Overow gets score by the community members on the basis of how useful or informative they are [3].…”

mentioning

confidence: 99%

Predicting citations from mainstream news, weblogs and discussion forums

Timilsina

¹

,

Davis

²

,

Michael

³

et al. 2017

Proceedings of the International Conference on Web Intelligence

6

0

View full text Add to dashboard Cite

The growth in the alternative digital publishing is widening the breadth of scholarly impact beyond the conventional bibliometric community. Thus, research is becoming more reachable both inside and outside of academic institutions and are found to be shared, downloaded and discussed in social media. In this study, we linked the scientic articles found in mainstream news, weblogs and Stack Overow to the citation database of peer-reviewed literature called Scopus. We then explored how standard graph-based inuence metrics can be used to measure the social impact of scientic articles. We also proposed the variant of Katz centrality metrics called EgoMet score to measure the local importance of scientic articles in its ego network. Later we evaluated these computed graph-based inuence metrics by predicting absolute citations. Our results of the prediction model describe 34% variance to predict citations from blogs and mainstream news and 44% variance to predict citations from Stack Overow.

show abstract

Peer Review Personas

Cited by 6 publications

References 0 publications

A multi-disciplinary perspective on emergent and future innovations in peer review

A multi-disciplinary perspective on emergent and future innovations in peer review

A multi-disciplinary perspective on emergent and future innovations in peer review

Predicting citations from mainstream news, weblogs and discussion forums

Contact Info

Product

Resources

About