Uses of artificial intelligence and machine learning in systematic reviews of education research

Abstract

With a growing researcher population, increased researcher interconnectivity, disciplinary fragmentation and lowered cost of manuscript processing and distribution, the speed and volume of scientific publishing is accelerating, both in terms of number of authors and in terms of the number of publications by each author. For systematic reviewers, this poses a challenge in the balance between not excluding too many possibly relevant studies ex ante and handling increasingly large corpora that result from document retrieval. Efforts to summarize and synthesize knowledge within or across domains are increasingly reliant on computational methods for document retrieval and analysis, but questions remain about the coverage and quality of automated review procedures. This article presents a workflow-centered framework for computational techniques for automated content analysis in the traditional workflow of systematic or scoping reviews. It provides guidelines for handling large document sets founded in best practice rather than relying on arbitrary search constraints to limit the number of publications under consideration and provides a discussion of where current efforts are headed and possible future developments using computational techniques, with a particular focus on the advent of large language models in ingesting and summarizing large knowledge corpora.