Editorial: Evidence on Questionable Research Practices: The Good, the Bad, and the Ugly

Abstract

Purpose Questionable research or reporting practices (QRPs) contribute to a growing concern regarding the credibility of research in the organizational sciences and related fields. Such practices include design, analytic, or reporting practices that may introduce biased evidence, which can have harmful implications for evidence-based practice, theory development, and perceptions of the rigor of science. Design/Methodology/Approach To assess the extent to which QRPs are actually a concern, we conducted a systematic review to consider the evidence on QRPs. Using a triangulation approach (e.g., by reviewing data from observations, sensitivity analyses, and surveys), we identified the good, the bad, and the ugly. Findings Of the 64 studies that fit our criteria, 6 appeared to find little to no evidence of engagement in QRPs and the other 58 found more severe evidence (91 %). Implications Drawing upon the findings, we provide recommendations for future research related to publication practices and academic training. Originality/value We report findings from studies that suggest that QRPs are not a problem, that QRPs are used at a suboptimal rate, and that QRPs present a threat to the viability of organizational science research.

Scientist as Subject: How Researcher Behaviors Influence Psychological Knowledge

“…In previous literature on QRP use, data fabrication is included as a questionable practice. Fabrication, along with falsification and plagiarism, have been previously labeled "FFP" and are not considered in this dissertation as they are not questionable but instead academically dishonest (Steneck, 2006 While there are some instances when QRP use may be justified, when they are used, they contribute to the false-positive rate observed in the published literature (Banks, Rogelberg, Woznyj, Landis, & Rupp, 2016;Fanelli, 2009). Not only does QRP use increase the number of false-positive findings (i.e.…”

Science’s reproducibility and replicability crisis: International business is not immune

“…In NHST, the tenability of a null hypothesis (i.e., no effect or relation) is primarily judged based on the observed p value associated with the test of the hypothesis, and values smaller than 0.05 are often judged as providing sufficient evidence to reject it (Bettis et al, 2016;Goldfarb & King, 2016). Of the many problems associated with this interpretation of p values, the most pernicious is that it motivates researchers to engage in a practice called ''p-hacking'' and to report ''crippled'' p values (see below) (Aguinis, Werner, Abbott, Angert, Park, & Kohlhausen, 2010;Banks, Rogelberg et al, 2016). For example, consider a researcher who interprets p = 0.0499 as sufficient evidence for rejecting the null hypothesis, and p = 0.0510 as evidence that the null hypothesis should be retained, and believes that journals are more likely to look favorably on rejected null hypotheses.…”

“…Clearly, it is difficult to make the case that research results are credible and useful if they are irreproducible and not replicable. Unfortunately, there is a proliferation of evidence indicating that lack of reproducibility and replicability are quite pervasive (e.g., Banks, Rogelberg, Woznyj, Landis, & Rupp, 2016;Cortina, Green, Keeler, & Vandenberg, 2016;Cuervo-Cazurra et al, 2016;Ioannidis, 2005;Open Science Collaboration, 2015;Schwab & Starbuck, 2017). Accordingly, as noted by Verbeke, Von Glinow, and Luo, ''… the IB discipline faces the challenges of remaining at par with the methodological standards in adjacent fields for validity, reliability, replicability and generalizability'' (Verbeke et al, 2017: 6).…”

A Call for Openness in Research Reporting: How to Turn Covert Practices Into Helpful Tools

“…It amounted to academic dissembling even though I knew it was commonly done" (Anonymous, 2015: 214). Banks, Rogelberg, Woznyj, Landis and Rupp (2016) examined 64 business studies and inferred that 91% of these studies showed evidence of covert, undesirable practices in the conduct or reporting of research.…”

“…These practices both distort evidence about the usefulness of theories and undermine confidence in the conclusions reported (as summarized in Table 1). The undesirable properties of these practices are wellestablished in the social sciences and the corresponding methods literature (Banks, Rogelberg, et al, 2016;Kepes, Bennett & McDaniel, 2014;Landis & Rogelberg, 2013;Schwab et al, 2011;Simmons et al 2011;Starbuck, 2016a).…”