A Machine Learning Approach for Semi-Automated Search and Selection in Literature Studies

Ros, Rasmus; Bjarnason, Elizabeth; Runeson, Per

doi:10.1145/3084226.3084243

Cited by 49 publications

(58 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the support for primary study selection using these tools is limited (e.g., to tasks such as assigning review jobs to multiple reviewers or to resolving disagreements). Hence, we planned to introduce machine learning to assist primary study selection in SE SLRs but before this paper is published, Ros et al [50] has achieved this in June 2017. While Ros'17 [50] provided a wide range of techniques to support both search and selection, it has several limitations such as (a) not comparing against state-of-the-art techniques from other domains (which are approaches discussed later in Section 3.1.2 and 3.1.3); (b) not considering any data balancing; (c) testing only on a single unpublished dataset.…”

Section: Software Engineering Toolsmentioning

confidence: 99%

Finding better active learners for faster literature reviews

Kraft

Menzies

2018

Empir Software Eng

View full text Add to dashboard Cite

Literature reviews can be time-consuming and tedious to complete. By cataloging and refactoring three state-of-the-art active learning techniques from evidence-based medicine and legal electronic discovery, this paper finds and implements FASTREAD, a faster technique for studying a large corpus of documents, combining and parametrizing the most efficient active learning algorithms. This paper assesses FASTREAD using datasets generated from existing SE literature reviews (Hall, Wahono, Radjenović, Kitchenham et al.). Compared to manual methods, FASTREAD lets researchers find 95% relevant studies after reviewing an order of magnitude fewer papers. Compared to other state-of-the-art automatic methods, FASTREAD reviews 20-50% fewer studies while finding same number of relevant primary studies in a systematic literature review.

show abstract

Section: Software Engineering Toolsmentioning

confidence: 99%

Finding better active learners for faster literature reviews

Kraft

Menzies

2018

Empir Software Eng

View full text Add to dashboard Cite

show abstract

“…In the following, we present the list of studies selected in this Systematic Mapping: S1 (Felizardo et al, 2010), S2 (Felizardo et al, 2011), S3 (Felizardo et al, 2012), S4 (Felizardo et al, 2014), S5 (Feng et al, 2017), S6 (Garcés et al, 2017), S7 (Malheiros et al, 2007), S8 (Mergel et al, 2015), S9 (Ouhbi et al, 2016), S10 (Piroi et al, 2015), S11 (Rizzo et al, 2017), S12 (Ros et al, 2017), S13 (Rúbio et al, 2016), S14 (Sellak et al, 2015), S15 (Tomassetti et al, 2011), S16 (Torres et al, 2013), S17 (Yu et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

Adoption of Machine Learning Techniques to Perform Secondary Studies: A Systematic Mapping Study for the Computer Science Field

Cairo

Carneiro

Silva

2019

Proceedings of the 21st International Conference on Enterprise Information Systems

View full text Add to dashboard Cite

Context: Secondary studies such as systematic literature reviews (SLR) have been used to collect and synthesize empirical evidence from relevant studies in several areas of knowledge, including Computer Science. However, secondary studies are time-consuming and require a significant effort from researchers. Goal: This paper aims to identify contributions derived from the adoption of machine learning (ML) techniques in Computer Science SLRs. Method: We performed a systematic mapping study querying well-known repositories and first found 399 studies as a result of applying the search string in each of the selected search engines. Following the research protocol, we analyzed titles and abstracts and applied inclusion, exclusion and quality criteria to finally obtain a set of 17 studies to be further analyzed. Results: The selected papers provided evidence of relevant contributions of the machine learning usage in performing secondary studies. We found that ML techniques have not been applied yet to all the stages of a SLR. Typically, the preferred stage to apply ML in an SLR is the study selection phase (typically the initial phase). For assessing the effectiveness of ML support while performing SLRs, researchers have provided a comparison either across different ML techniques tested or between manual and ML-supported SLRs. Conclusion: There is significant evidence that the use of machine learning applied to SLR activities (especially the study selection activity) in Computer Science is feasible and promising, and the findings can be potentially extended to other research fields. Also, there is a lack of studies exploring ML techniques for other stages than study selection.

show abstract

“…Kuhrmann et al [44] provide recommendations specifically for the general study design, data collection, and study selection procedures. Ros et al [45] propose a machine learning approach to classifying papers by leveraging human experts, who iteratively validate sets of publications produced by a classifier. Conversely, EDAM does not require experts to manually examine research papers, but only to review a taxonomy of research areas.…”

Section: Topic Trend Analysismentioning

confidence: 99%

The evolution of IJHCS and CHI: A quantitative analysis

Mannocci

Osborne²,

Motta³

2019

International Journal of Human-Computer Studies

View full text Add to dashboard Cite

In this paper we focus on the International Journal of Human-Computer Studies (IJHCS) as a domain of analysis, to gain insights about its evolution in the past 50 years and what this evolution tells us about the research landscape associated with the journal. To this purpose we use techniques from the field of Science of Science and analyse the relevant scholarly data to identify a variety of phenomena, including significant geopolitical patterns, the key trends that emerge from a topic-centric analysis, and the insights that can be drawn from an analysis of citation data. Because the area of Human-Computer Interaction (HCI) has always been a central focus for IJHCS, we also include in the analysis the CHI conference, which is the premiere scientific venue in HCI. Analysing both venues provides more data points to our study and allows us to consider two alternative viewpoints on the evolution of HCI research.

show abstract

A Machine Learning Approach for Semi-Automated Search and Selection in Literature Studies

Cited by 49 publications

References 27 publications

Finding better active learners for faster literature reviews

Finding better active learners for faster literature reviews

Adoption of Machine Learning Techniques to Perform Secondary Studies: A Systematic Mapping Study for the Computer Science Field

The evolution of IJHCS and CHI: A quantitative analysis

Contact Info

Product

Resources

About