Extracting Information from Experimental Software Engineering Papers

Recommended Steps for Thematic Synthesis in Software Engineering

“…We recommend a structured reading technique as proposed by Cruzes et al to explore the evidence of a systematic review [17]. Using this technique, the reviewer follows a procedure for identifying context information and the findings of the paper.…”

“…Although not explicitly stated in the text of the paper, relationships visually expressed in figures can also be extracted and translated into textual form. When identifying a statement that could potentially be a finding, the following questions might help [17]:…”

Research synthesis in software engineering: A tertiary study

“…Synthesis of findings across diverse study designs is far from simple and is likely to be aimed at identifying recurring themes in the studies or common contextual factors [9,11]. SRs conducted with respect to the determination of why study results differ (as they are likely to do), and the evaluation of the potentially contrasting insights from qualitative and quantitative studies will generally be more helpful in SE than those that focus on identifying average effects.…”

Automatic Results Identification in Software Engineering Papers. Is it Possible?

“…The task reading a paper and correctly identify the sentences that represent the results of the study, for example, presents itself as a complex task even for human experts [4]. Cruzes et.…”

“…al report an experiment in which some Software Testing papers were distributed to groups of graduate students so that they identify the sentences that represented the results of the studies described. At the end of the experiment, it was observed that, on average, only 50% of the existing results have been correctly located [4]. Another experiment provided a set of health area papers to be analyzed by three different experts.…”