If It Were Only That Easy: The Use of Meta-Analytic Research by Organizational Scholars

DeSimone, Justin A.; Köhler, Tine; Schoen, Jeremy L.

doi:10.1177/1094428118756743

Cited by 33 publications

(22 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is clear that the workplace deviance literature is burgeoning, but there is still much room for additional research. Next, we describe actionable opportunities to incrementally improve areas of limited knowledge so others can meaningfully utilize the information our meta-analysis provides as they design future studies (Carlson & Ji, 2011; DeSimone, Köhler, & Schoen, 2018).…”

Section: Discussionmentioning

confidence: 99%

A Meta-Analysis of Interpersonal and Organizational Workplace Deviance Research

Mackey

McAllister²,

Ellen

et al. 2019

Journal of Management

121

120

View full text Add to dashboard Cite

Workplace deviance research has expanded rapidly over the past decade. Despite the expansive body of research available, we have an incomplete understanding of the measurement, magnitude, and direction of relationships within workplace deviance’s nomological network. We draw from 235 empirical samples of data (k = 235, N = 66,990) to conduct random-effects meta-analyses of interpersonal (k = 156, N = 42,239) and organizational (k = 206, N = 60,008) workplace deviance research so we can build the solid foundation necessary to advance the conversation in this literature. We use an exploratory meta-analytic approach and the horizontal contrasting method of theory elaboration to provide a nuanced understanding of the relationship between interpersonal and organizational deviance (ρ = .67, k = 110, N = 30,426) as well as determine the magnitude and generalizability (i.e., external validity) of relationships within their nomological networks. We find some evidence of differences (i.e., heterogeneity) in the measurement and magnitude of relationships across contexts. Overall, we conduct a state-of-the-art meta-analysis that leverages contemporary meta-analytic techniques and the extensive body of empirical workplace deviance research available to improve our understanding of the measurement of workplace deviance and relationships within its nomological network. Our study instigates new debates about the completeness of our understanding of workplace deviance and provides the empirical building blocks necessary for this literature to reach its unrealized potential.

show abstract

Section: Discussionmentioning

confidence: 99%

A Meta-Analysis of Interpersonal and Organizational Workplace Deviance Research

Mackey

McAllister²,

Ellen

et al. 2019

Journal of Management

121

120

View full text Add to dashboard Cite

show abstract

“…Meta-analyses are overwhelmingly used to understand what is the overall (i.e., average) size of the relationship between variables across primary-level studies (DeSimone et al, 2019;Carlson & Ji, 2011). However, meta-analysis is just as useful, if not more so, to understand when and where a relationship is likely to be stronger or weaker (Aguinis, Pierce, et al, 2011).…”

Section: Stage 3: Data Analysismentioning

confidence: 99%

The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews

2021

View full text Add to dashboard Cite

Meta-analyses summarize a field’s research base and are therefore highly influential. Despite their value, the standards for an excellent meta-analysis, one that is potentially award-winning, have changed in the last decade. Each step of a meta-analysis is now more formalized, from the identification of relevant articles to coding, moderator analysis, and reporting of results. What was exemplary a decade ago can be somewhat dated today. Using the award-winning meta-analysis by Stahl et al. (Unraveling the effects of cultural diversity in teams: A meta-analysis of research on multicultural work groups. Journal of International Business Studies, 41(4):690–709, 2010) as an exemplar, we adopted a multi-disciplinary approach (e.g., management, psychology, health sciences) to summarize the anatomy (i.e., fundamental components) of a modern meta-analysis, focusing on: (1) data collection (i.e., literature search and screening, coding), (2) data preparation (i.e., treatment of multiple effect sizes, outlier identification and management, publication bias), (3) data analysis (i.e., average effect sizes, heterogeneity of effect sizes, moderator search), and (4) reporting (i.e., transparency and reproducibility, future research directions). In addition, we provide guidelines and a decision-making tree for when even foundational and highly cited meta-analyses should be updated. Based on the latest evidence, we summarize what journal editors and reviewers should expect, authors should provide, and readers (i.e., other researchers, practitioners, and policymakers) should consider about meta-analytic reviews.

show abstract

“…However, these are serious issues that reviewers need to weigh heavily in their recommendation about the work. Given grave methodological flaws, a piece of work might communicate, in the best case, meaningless findings, and, in the worst case, wrong findings, which may then get perpetuated as the work gets cited (e.g., DeSimone, Köhler, & Schoen, 2019). As a first step, reviewers should probe the author for answers (c) (e.g., "There are three missing degrees of freedom in the CFA.…”

Section: Competency Framework For Peer Review 30mentioning

confidence: 99%

Supporting robust, rigorous, and reliable reviewing as the cornerstone of our profession: Introducing a competency framework for peer review

Köhler

González-Morales

Banks

et al. 2020

Ind. Organ. Psychol.

Self Cite

View full text Add to dashboard Cite

Peer review is a critical component toward facilitating a robust science in industrial and organizational (I-O) psychology. Peer review exists beyond academic publishing in organizations, university departments, grant agencies, classrooms, and many more work contexts. Reviewers are responsible for judging the quality of research conducted and submitted for evaluation. Furthermore, they are responsible for treating authors and their work with respect, in a supportive and developmental manner. Given its central role in our profession, it is curious that we do not have formalized review guidelines or standards and that most of us never receive formal training in peer reviewing. To support this endeavor, we are proposing a competency framework for peer review. The purpose of the competency framework is to provide a definition of excellent peer reviewing and guidelines to reviewers for which types of behaviors will lead to good peer reviews. By defining these competencies, we create clarity around expectations for peer review, standards for good peer reviews, and opportunities for training the behaviors required to deliver good peer reviews. We further discuss how the competency framework can be used to improve peer reviewing and suggest additional steps forward that involve suggestions for how stakeholders can get involved in fostering high-quality peer reviewing.

show abstract

If It Were Only That Easy: The Use of Meta-Analytic Research by Organizational Scholars

Cited by 33 publications

References 89 publications

A Meta-Analysis of Interpersonal and Organizational Workplace Deviance Research

A Meta-Analysis of Interpersonal and Organizational Workplace Deviance Research

The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews

Supporting robust, rigorous, and reliable reviewing as the cornerstone of our profession: Introducing a competency framework for peer review

Contact Info

Product

Resources

About