Topics, publication patterns, and reporting quality in systematic reviews in language education. Lessons from the international database of education systematic reviews (IDESR)

Abstract

The International Database of Education Systematic Reviews (IDESR.org) contains summary records of published systematic reviews in education and protocols for unpublished reviews and reviews in preparation. During its pilot phase, IDESR is concentrating exclusively on curating systematic reviews in language education. IDESR makes ready access to extant evidence syntheses for researchers, who can use this information to assess the strength of the warrant for any proposed new primary research and/or additional evidence syntheses. By using IDESR to publish review protocols prospectively, review authors commit to high standards of transparency and rigour in producing their research. We have used the data held in IDESR to assess the topics, publication patterns, and reporting quality in the language education literature. We found (i) that language education has seen exponential growth in systematic reviews of research; (ii) that a variety of topics have been addressed, but those related to educational technology have dominated; (iii) that reviews are published in a wide range of outlets, going beyond language education journals; and (iv) that there is room for improvement in the quality of reporting evidence syntheses in language education.

Setting research priorities for applied cognitive sciences—What do teachers want from research?

“…In conclusion, this PSP has highlighted the importance of adopting an approach to the study of cognitive sciences that allows for the description and analysis of more complex relationships between strategies, their implementation in lessons and curricula and how they interact with other evidence‐informed teaching approaches as well as student motivation and agency. Conducting systematic reviews on these questions will be an important first step to guide decisions about new primary research (Chalmers et al., 2023).…”

Reducing ‘avoidable research waste’ in applied linguistics research: Insights from healthcare research

“…Similarly, a systematic reviewer cannot reliably or comprehensively extract key details from a study nor assess its methodological quality when it is poorly reported. A recent audit of 307 systematic reviews in language education (Chalmers et al, 2023) and 120 meta-analyses of L2 experimental and quasi-experimental studies (Vuogan & Li, 2023) have revealed notable omissions in research reporting (e.g., eligibility criteria, primary study/participant sample characteristics, articulation of the research questions, risk of bias/trustworthiness, statistical model selection, etc.). When important information that allows readers to interpret a review and its findings are absent from supposedly authoritative syntheses of a field of research, readers are denied the opportunity to accurately assess the state of knowledge and strength of evidencea wasted opportunity with repercussions for subsequent research that relies on it for information.…”