Publication of research on controversial topics: The early acceptance procedure

Armstrong, J. Scott

doi:10.1016/0169-2070(95)00626-5

Cited by 15 publications

(18 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore it is of utmost importance to take measures against it. A possible measure would be to encourage scientific journals to implement alternative modes of the review process, such as the early acceptance procedure as described by Armstrong (1996) which is based on the reviewing of the design of a study and can be done before the study is conducted (for a more detailed description of the early acceptance procedure, see Armstrong 1996). Future studies should aim to find further startingpoints and methods to reduce the influence of confirmation bias within the scientific review process.…”

Section: Overall Discussionmentioning

confidence: 98%

“…Academic psychologists rated studies that report findings consistent with their prior beliefs more favorably than studies reporting findings inconsistent with their previous beliefs. This is especially true for research with the potential to produce controversial findings, which on the other hand is vital for the progress in psychological science (Armstrong 1996). In the worst case this could lead to the suppression of new information over a long period, because of often harsh peer review given to research presenting controversial results (Horrobin 1990).…”

Section: Overall Discussionmentioning

confidence: 99%

“…However, assuming that the research question is relevant, the experimental design adequate and the data are clearly and comprehensively described, the found results should be of importance to the scientific community and should not be viewed prejudiciallyregardless of whether they conform to current theoretical predictions (Mahoney 1977). One could even say that research with the potential to produce controversial findings is especially important to progress in the sciences (Armstrong 1996). But scientific innovators often meet with resistance from the scientific community (Armstrong 1996; also see Horrobin 1990 for numerous examples of harsh peer review given to research presenting controversial results).…”

Section: Confirmation Bias In Academiamentioning

confidence: 92%

“…One could even say that research with the potential to produce controversial findings is especially important to progress in the sciences (Armstrong 1996). But scientific innovators often meet with resistance from the scientific community (Armstrong 1996; also see Horrobin 1990 for numerous examples of harsh peer review given to research presenting controversial results). Therefore, confirmation bias may be especially harmful to objective evaluations regarding nonconforming results, since biased individuals might regard opposing evidence to be weak in principle and give little serious thought to revising those beliefs (Koehler 1993).…”

Section: Confirmation Bias In Academiamentioning

confidence: 98%

See 3 more Smart Citations

Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further Evidence of a Confirmation Bias Within Scientific Psychology

2010

View full text Add to dashboard Cite

The present paper investigated whether academic psychologists show a tendency to rate the quality and appropriateness of scientific studies more favorably when results and conclusions are consistent with their own prior beliefs (i.e., confirmation bias). In an online experiment, 711 psychologists completed a questionnaire (e.g., about their belief in astrology) and evaluated research that was presented in form of a short abstract in which 40 different behaviors (e.g., alcohol consumption, willingness to share money) have been tried to be predicted. The research to be evaluated varied on three dimensions which were all manipulated between subjects: (1) the predictors of the 40 behaviors (either Big Five or astrological factors), (2) the methodological quality of the study (low, medium, high), and (3) the results and subsequent conclusion of the study (confirmation or disconfirmation of the hypotheses). Factor-analyzed scores of participants' ratings on 8 scales, resulting in 2 factors termed quality and appropriateness, served as dependent measures. The main result of the study is a two-way interaction: Psychologists tended to evaluate results qualitatively higher when they conformed to their own prior expectations, as in this case, when astrological hypotheses were disconfirmed.

show abstract

Section: Overall Discussionmentioning

confidence: 98%

Section: Overall Discussionmentioning

confidence: 99%

Section: Confirmation Bias In Academiamentioning

confidence: 92%

Section: Confirmation Bias In Academiamentioning

confidence: 98%

See 2 more Smart Citations

Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further Evidence of a Confirmation Bias Within Scientific Psychology

2010

View full text Add to dashboard Cite

show abstract

“…That new research should be judged by those that have established and furthered past research would seem likely to bias against innovation. This may be especially problematic in areas with dynamic methodologi-( ) cal frontiers Armstrong, 1996 . More generally, scholarly reputations are often grounded in particular positions or traditions that might be jeopar-( ) dized by new approaches or ideas see also Archer, 1996 .…”

Section: The Conventional Critiquementioning

confidence: 99%

Making Accounting Knowledge: Peering at Power

Fogarty

Ravenscroft

2000

Critical Perspectives on Accounting

View full text Add to dashboard Cite

Peer review and secrecy in the “Information Age”

Weicher

2008

Proc of Assoc for Info

View full text Add to dashboard Cite

The subject of peer review has come under increased scrutiny partially due to high profile cases of fraud and other misinformation appearing in academic journals. At the same time, new publishing technologies have made it possible to experiment with new forms of peer review, while the "Open Archives Initiative" has the potential to allow authors to bypass the traditional peer review process entirely. In this paper, I examine some rationales given for the need for secrecy in peer review, especially as it relates to the perceived need for anonymity on the part of reviewers and sometimes even authors. I will propose a basic framework for secrecy and transparency as it relates to the peer review process of academic journals. IntroductionIn December 2006, there was a debate about peer review featured prominently on the website of the journal Nature. In addition to a series of articles discussing various systems of peer review; its effect on the quality and value of research; research ethics; new technical implementations of peer review; and a series of individual perspectives and reflections on the subject; it includes an experiment in "open" peer review. Authors were given the option of posting their manuscripts for open peer review at the same time they submitted their work for the more traditional, confidential peer review process. For authors who elected to participate in the experiment in open peer review, comments on their research were solicited from qualified members of the public. Those who wished to comment could do so on the moderated forum on the Nature website. Unlike traditional peer review, anonymous comments were not allowed. The self-selected reviewers were required to give both their name and institutional affiliation. The stated purpose of this experiment was to "measure the level of participation among authors and the quality of comments received by members of the specialist community who are not the selected peer reviewers of the manuscripts concerned." (Nature website, 2006).Despite some initial enthusiasm, the four month experiment met with mixed results. Only five percent of authors chose to post their papers on the open server for comments, perhaps due to the fear of being "scooped". Except for a handful of relative highly commented on papers, there was a dearth of substantive comments despite statistics that showed a high level of web traffic.Biology Direct is an example of a journal that has gone beyond the experimentation phase, and formally implemented a system of open peer review. An online journal that began publication in January 2006, the editorial policy requires that reviewer comments, along with the author's response, are published alongside the article. The main hurdle to authors to overcome in order to qualify for publication is that three members of the editorial board must become sufficiently interested in the research to agree to act as reviewers or to solicit outside reviewers. Once this requirement is met, the policy even allows for publication of papers that have r...

show abstract

Publication of research on controversial topics: The early acceptance procedure

Cited by 15 publications

References 13 publications

Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further Evidence of a Confirmation Bias Within Scientific Psychology

Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further Evidence of a Confirmation Bias Within Scientific Psychology

Making Accounting Knowledge: Peering at Power

Peer review and secrecy in the “Information Age”

Contact Info

Product

Resources

About