Improving the Design and Use of Meta Analyses in Evidence-Based Practice

Becker, Betsy Jane

doi:10.17125/svp2016.ch10

Cited by 11 publications

(9 citation statements)

References 32 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has the added advantage of ascertaining empirically the degree to which scales that are supposedly measuring the same constructs (by their titles) actually do so (See Becker's example of the self-efficacy measures used in the Brown and Ryan Krane meta-analysis). I also concur with Becker's (2017) suggestion that we create a shared database of information on career intervention effect sizes. This may provide a valuable vehicle for addressing questions of construct redundancy (in addition to its other benefits-See Becker, 2017).…”

Section: Future Need #1: We Need To Identify Core Outcome Constructssupporting

confidence: 72%

Meta-Analysis and Evidence-Based Career Practice: Current Status and Future Directions

Brown

2017

Integrating Theory, Research, and Practice in Vocational Psychology: Current Status and Future Directions

View full text Add to dashboard Cite

Section: Future Need #1: We Need To Identify Core Outcome Constructssupporting

confidence: 72%

Meta-Analysis and Evidence-Based Career Practice: Current Status and Future Directions

Brown

2017

Integrating Theory, Research, and Practice in Vocational Psychology: Current Status and Future Directions

View full text Add to dashboard Cite

“…For example, studies A and B may vary simply because of sampling error, differences in the quality of the research, attributes of the participants, intervention variations, or differences among outcome measures (Tickle-Degnen, 2001). As both Becker (2017) and Brown (2017) identify, the quality of the outcome measures used in a meta-analysis is critical to the conclusions that can be drawn from the results. In conducting a meta-analysis, the researcher typically conducts a statistical test of homogeneity to determine whether the differences in the effect sizes are likely due to sampling error (Lipsey & Wilson, 2001).…”

mentioning

confidence: 99%

“…Hence, analyses of moderator variables allow the researcher to go from general findings to more specific results. For a discussion of moderator analyses and for ways to expand on moderator analyses, please see Becker (2017).…”

mentioning

confidence: 99%

Conducting General versus Population and Setting-Specific Meta Analyses

Whiston

2017

Integrating Theory, Research, and Practice in Vocational Psychology: Current Status and Future Directions

View full text Add to dashboard Cite

Meta-analysis is an increasingly popular technique that systematically and statistically summarizes the results from research studies in a particular area (Lipsey & Wilson, 2001). Meta-analytic studies can be particularly useful to practitioners because they combine the results from various studies and eliminate the drudgery of sifting through the findings of multiple individual studies. Although meta-analysis can be used to summarize research in multiple areas (e.g., correlational studies), I am going to restrict my comments to intervention studies with a focus on career interventions. Specifically, this chapter will address the issues surrounding conducting general meta-analyses versus population and setting-specific meta-analyses in the career area.With meta-analysis, a researcher calculates an effect size for each study, typically by subtracting the control group mean from the treatment group mean and dividing by the pooled standard deviation of the groups. Therefore, if the treatment group mean is larger than the control group, the effect size is positive and reflects the degree or magnitude of the treatment group's effect. These effect sizes are weighted, on factors such as sample size, and are combined to produce an average effect size for the intervention studies. The content of the combined intervention studies is important in this process, along with whether it is better to more narrowly restrict studies to specific populations (e.g., AfricanAmericans) or settings (e.g., career development centers) or to more broadly include studies, which will be labeled general meta-analyses.Certainly there are advantages to having more specific meta-analyses. For example, a career counselor working with African American clients could search databases such as PSYCINFO for meta-analyses of career interventions with African Americans. A recent search of PSYCINFO with the keyword meta-analysis revealed over 16,000 studies. It would be difficult for the career counselor to review each of the studies in the hopes of identifying a meta-analysis pertinent to career counseling. Thereby, a clinician could simply enter the terms African American and career interventions and theoretically identify a meta-analysis, if such a metaanalysis existed, that could inform the clinician on the types of career interventions associated with larger effect sizes. The clinician could then adopt those interventions shown to be most effective with African American clients and provide evidence-based practice. Although such a meta-analysis does not currently exist, the desire for such information is evident as many researchers have called for an exploration of the efficacy of career interventions with ethnic and racial minorities (Fouad, & Kantamneni, 2008;Leong & Flores, 2015). Not only is there a need for specific information regarding the effectiveness of career intervention with racial and ethnic minority clients, but there may also be a need for specific

show abstract

“…First, Nadya Fouad (2017) examines the overall role of theory and how it can guide efforts to minimize the gap between science and practice. Next, chapters by Steve Brown (2017), Susan Whiston (2017), and Betsy Becker (2017) offer various perspectives on the state of the art in conducting meta-analyses on the outcomes of career development interventions. Finally, Scott Solberg (2017) offers a personal reflection on efforts to link vocational psychology scholarship to specific policy initiatives that require interdisciplinary collaborations.…”

mentioning

confidence: 99%

“…Building on the knowledge from prominent meta-analyses of career outcomes concluding that career interventions are effective, there is a critical need to conduct more primary intervention research (Whiston, 2017), especially across diverse samples (Brown, 2017) involving a range of presenting concerns and settings. Becker (2017) featured the MUTOS method that enables researchers to examine sources of variability in outcomes across studies, including potential moderators involving client characteristics and features of interventions. Indeed, coordinated efforts to understand what works for whom (Frank & Frank, 1993) under what conditions should be applied more vigorously to career development concerns.…”

mentioning

confidence: 99%

Summary of Key Elements in Using Theory and Research to Improve Evidence-Based Practice

Rottinghaus

2017

Integrating Theory, Research, and Practice in Vocational Psychology: Current Status and Future Directions

View full text Add to dashboard Cite

This section covered a broad range of concerns related to the integration of theory, research, and practice, including implications for collaboration and policy. Theory-driven work is paramount in vocational psychology and should guide research and practice initiatives. This set of chapters features several ongoing challenges and opportunities that can facilitate the translation of scholarship to inform the development of evidence-based interventions that matter to practice settings and policy decisions. First, Nadya Fouad (2017) examines the overall role of theory and how it can guide efforts to minimize the gap between science and practice. Next, chapters by Steve Brown (2017), Susan Whiston (2017), and Betsy Becker (2017) offer various perspectives on the state of the art in conducting meta-analyses on the outcomes of career development interventions. Finally, Scott Solberg (2017) offers a personal reflection on efforts to link vocational psychology scholarship to specific policy initiatives that require interdisciplinary collaborations. This chapter summarizes important highlights and themes from these chapters, and concludes with five strategies for purposefully connecting theory and research to strengthen evidence-based practice.Fouad (2017) highlights personal reflections on her experiences bridging gaps between theory, research, and practice. As an important reminder to students, even distinguished leaders in our field once struggled to know how to apply theories to working with actual clients. Fortunately, theories can work as guiding maps to assist counselors in conceptualizing clients' concerns. Swanson and Fouad's (2015) career text focuses on applying career theories to case studies beyond presenting theories as separate from application with clients. Additionally, Fouad (2017) noted that ongoing attempts to bring together researchers and practitioners can be difficult and requires collaborations involving teams of experts from practice and science settings. Drawing on her experiences editing leading applied psychology journals, Fouad shared examples of creative attempts to bridge the science-practice gap. For example, she initiated the Practitioner Forum during her time as the editor of The Counseling Psychologist. Again, hearing a frank discussion that normalizes the challenges our leading scholars face can help future professionals have the courage to build on these efforts by modifying approaches to transferring knowledge between science and practice, guided by theory.A crucial line of research on career-related outcomes has been the series of initial meta-analyses that demonstrated the effectiveness of career interventions (Oliver & Spokane, 1988;Spokane & Oliver, 1983). Brown (2017) elaborated on three recent meta-analyses that have built upon Oliver and Spokane's fundamental work, including Whiston, Sexton, and Lasoff's (1998) study that found individual counseling was more effective than group modalities. Brown and Ryan Krane (2000) identified common critical ingredients

show abstract

Improving the Design and Use of Meta Analyses in Evidence-Based Practice

Cited by 11 publications

References 32 publications

Meta-Analysis and Evidence-Based Career Practice: Current Status and Future Directions

Meta-Analysis and Evidence-Based Career Practice: Current Status and Future Directions

Conducting General versus Population and Setting-Specific Meta Analyses

Summary of Key Elements in Using Theory and Research to Improve Evidence-Based Practice

Contact Info

Product

Resources

About