Identifying Barriers to the Systematic Literature Review Process

Abstract

Conducting a systematic literature review (SLR) is difficult and time-consuming for an experienced researcher, and even more so for a novice graduate student. With a better understanding of the most common difficulties in the SLR process, mentors will be better prepared to guide novices through the process. This understanding will help researchers have more realistic expectations of the SLR process and will help mentors guide novices through its planning, execution, and documentation phases. Consequently, the objectives of this work are to identify the most difficult and time-consuming phases of the SLR process. Using data from two sources -52 responses to an online survey sent to all authors of SLRs published in software engineering venues and qualitative experience reports from 8 PhD students who conducted SLRs as part of a course -we identified specific difficulties related to each phase of the SLR process. Our findings highlight the importance of planning, teamwork, and mentoring by an experienced researcher throughout the process. The paper also identifies implications for the teaching of the SLR process.

A Model-Based Approach to Systematic Review of Research Literature

“…Knowledge intensiveness, tediousness and errorproneness are the key factors cited for low adoption of SR in SE. Several tertiary reviews [24,1,7,8,10] are congruent with these observations.…”

Tools to support systematic reviews in software engineering

“…Only one tool, SLR-Tool, assisted this stage effectively. In addition, support for the search process (a frequently stressed issue within the community [5,6,7,8]), is largely absent. SLRTOOL is the only tool that provides an internal search facility (i.e.…”

“…Despite having a well-defined process, SRs are quite labour intensive and error prone [5,6,7,8] making them prime candidates for automated support. A number of tools to support the SR process were identified through a recent mapping study which found a predominance of visualisation and text mining techniques to support study selection, data extraction and data synthesis [9].…”

Outcomes of a community workshop to identify and rank barriers to the systematic literature review process

“…In our previous work (described in more detail in Section 2.2) [2], we identified barriers that hinder the SLR process. Based on our analysis of data from three sources: (1) our own experiences conducting SLRs, (2) a graduatelevel course on SLRs, and (3) a survey of authors of SLRs published in SE venues, we identified the aspects of the SLR process that were most difficult, most time-consuming, and most in need of tool support.…”