Evaluating and Interpreting Research Syntheses in Adult Learning and Literacy

Cooper, HM

doi:10.1037/e549792010-001

Cited by 17 publications

(11 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most practice in this area is rudimentary, largely relying on Cohen's (1988) rules of thumb for what constitutes a ''small,'' ''medium,'' or ''large'' effect size. Despite what these labels imply, Cohen did not intend them to be used as indicators of practical significance (Cooper, 2007). Instead he developed them to provide an indication of the magnitude of the effect that one might expect to find in order to inform prospective power analysis.…”

Section: Present and Interpret Effect Sizesmentioning

confidence: 99%

Life After NHST: How to Describe Your Data Without “p-ing” Everywhere

Valentine

Aloe

Lau

2015

Basic and Applied Social Psychology

View full text Add to dashboard Cite

Section: Present and Interpret Effect Sizesmentioning

confidence: 99%

Life After NHST: How to Describe Your Data Without “p-ing” Everywhere

Valentine

Aloe

Lau

2015

Basic and Applied Social Psychology

View full text Add to dashboard Cite

“…Meta-analysis is also able to identify gaps between available studies and to suggest future research directions or even formulate research agendas. As a systematic review, meta-analysis employs statistical methods to integrate and summarize primary studies on a particular topic by comprehensively locating research studies using “organized, transparent, and replicable procedures at each step in the process” (Cooper, 2007: 1). Meta-analysis, like most primary studies and any form of systematic review, follows similar steps: topic formulation, treatment design, sampling, data collection, data analysis, and reporting of results.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Nowadays though, Glass’s model of research synthesis is no longer considered appropriate, as new methods for retrieving, integrating, and interpreting research findings have been developed (Cooper, 2007). Researchers have argued that the decision rules outlined by the originators of meta-analysis such as Glass, McGaw and Smith (1981) back in the 1980s should be modified and expanded as new meta-analytic methodologies are developed (Cooper, 2007). This need to revise the practice of meta-analysis has arisen also due to the many unanticipated results arrived at by meta-analyses on the same topic.…”

Section: Introductionmentioning

confidence: 99%

Transparency of reporting in CALL meta-analyses between 2003 and 2015

Lin

Chen

Liou

2017

ReCALL

View full text Add to dashboard Cite

Since its introduction by Glass in the 1970s, meta-analysis has become a widely accepted and the most preferred approach to conducting research synthesis. Overcoming the weaknesses commonly associated with traditional narrative review and vote counting, meta-analysis is a statistical method of systematically aggregating and analyzing empirical studies by following well-established procedures. The findings of a meta-analysis, when appropriately conducted, are able to inform important policy decisions and provide practical pedagogical suggestions. With the growing number of publications employing meta-analysis across a wide variety of disciplines, it has received criticism due to its inconsistent findings derived from multiple meta-analyses in the same research domain. These inconsistencies have arisen partly due to the alternatives available to meta-analysts in each major meta-analytic procedure. Researchers have therefore recommended transparent reporting on the decision-making for every essential judgment call so that the results across multiple meta-analyses become replicable, consistent, and interpretable. This research explored the degree to which meta-analyses in the computer-assisted language learning (CALL) discipline transparently reported their decisions in every critical step. To achieve this aim, we retrieved 15 eligible meta-analyses in CALL published between 2003 and 2015. Features of these meta-analyses were extracted based on a codebook modified from Cooper (2003) and Aytug, Rothstein, Zhou and Kern (2012). A transparency score of reporting was then calculated to examine the degree to which these meta-analyses are compliant with the norms of reporting as recommended in the literature. We then discuss the strengths and weaknesses of the methodologies and provide suggestions for conducting quality meta-analyses in this domain.

show abstract

“…The first stage in research synthesis involves defining the review question and determining what research evidence is germane to the problem or hypothesis of interest (Cooper, ). Understanding the “big picture” can stimulate debate on priority areas where evidence can play a role in shaping policy and practice and where it is less likely to do so.…”

Section: Scoping the Reviewmentioning

confidence: 99%

Using logic models to capture complexity in systematic reviews

et al. 2011

View full text Add to dashboard Cite

Logic models have long been used to understand complex programs to improve social and health outcomes. They illustrate how a program is designed to achieve its intended outcomes. They also can be used to describe connections between determinants of outcomes, for example, low high-school graduation rates or spiraling obesity rates, thus aiding the development of interventions that target causal factors. However, these models have not often been used in systematic reviews. This paper argues that logic models can be valuable in the systematic review process. First, they can aid in the conceptualization of the review focus and illustrate hypothesized causal links, identify effect mediators or moderators, specify intermediate outcomes and potential harms, and justify a priori subgroup analyses when differential effects are anticipated. Second, logic models can be used to direct the review process more specifically. They can help justify narrowing the scope of a review, identify the most relevant inclusion criteria, guide the literature search, and clarify interpretation of results when drawing policy-relevant conclusions about review findings. We present examples that explain how logic models have been used and how they can be applied at different stages in a systematic review. Copyright © 2011 John Wiley & Sons, Ltd.

show abstract

Evaluating and Interpreting Research Syntheses in Adult Learning and Literacy

Cited by 17 publications

References 51 publications

Life After NHST: How to Describe Your Data Without “p-ing” Everywhere

Life After NHST: How to Describe Your Data Without “p-ing” Everywhere

Transparency of reporting in CALL meta-analyses between 2003 and 2015

Using logic models to capture complexity in systematic reviews

Contact Info

Product

Resources

About