On the Vocabulary Agreement in Software Issue Descriptions

Chaparro, Oscar; Florez, Juan Manuel; Marcus, Andrian

doi:10.1109/icsme.2016.44

Cited by 9 publications

(18 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are two different types of duplicate videos that exist in the corpus: (i) those videos that are a duplicate of the query (the Same Bug group), and (ii) those videos which are duplicates of each other, but are not a duplicate of the query (the Different Bug group). This second type of duplicate video is represented by bug reports marked as duplicates in the issue tracker and their corresponding master reports [35,86,93]. Each non-duplicate video reports a distinct bug.…”

Section: B Duplicate Detection Tasksmentioning

confidence: 99%

“…We executed each Ta n g o configuration on the 4,860 duplicate detection tasks and measured its effectiveness using standard metrics used in prior text-based duplicate bug detection research [35,86,93]. For each task, we compare the ranked list of videos produced by TANGO and the expected duplicate videos from the ground truth.…”

Section: Tango's Execution and Effectiveness Measurementmentioning

confidence: 99%

“…Precision@k is the proportion of the top-k returned videos that are duplicates according to the ground truth. We also computed HlT@k (aka Recall Rate@k [35,86,93]), which is the proportion of tasks that are successful for the cut point k of the ranked list. A task is successful if at least one duplicate video is found in the top-k results returned by TANGO.…”

Section: Tango's Execution and Effectiveness Measurementmentioning

confidence: 99%

“…To verify this hypothesis, we measured the shared vocabulary of duplicate and non-duplicate video pairs, similarly to Chaparro et al's analysis of textual bug reports [35]. We formed unique pairs of duplicate and non-duplicate videos from all the videos collected for all six apps.…”

Section: Appmentioning

confidence: 99%

“…III-E), we investigated the effort required for experienced programmers to identify duplicate video-based bug reports and found that participants reported a range of difficulties for the task (e.g., a single change of one step can result in two very similar looking videos showing entirely different bugs), and spent about 20 seconds of comprehension effort on average per video viewed. If this effort is extrapolated to the large influx of bug reports that could be collected on a daily basis [27,35,86,87], it illustrates the potential for the excessive effort associated with video-based duplicate bug detection. This is further supported by the plans of a large company that supports crowd-sourced bug reporting (name omitted for anonymity), which we contacted as part of eliciting the design goals for this research, who stated that they anticipate increasing developer effort in managing videobased reports and that they are planning to build a feature in their framework to support this process.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

It Takes Two to Tango: Combining Visual and Textual Information for Detecting Duplicate Video-Based Bug Reports

Cooper

Bernal-Cárdenas

Chaparro

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

When a bug manifests in a user-facing application, it is likely to be exposed through the graphical user interface (GUI). Given the importance of visual information to the process of identifying and understanding such bugs, users are increasingly making use of screenshots and screen-recordings as a means to report issues to developers. However, when such information is reported en masse, such as during crowd-sourced testing, managing these artifacts can be a time-consuming process. As the reporting of screen-recordings in particular becomes more popular, developers are likely to face challenges related to manually identifying videos that depict duplicate bugs. Due to their graphical nature, screen-recordings present challenges for automated analysis that preclude the use of current duplicate bug report detection techniques. To overcome these challenges and aid developers in this task, this paper presents TA N G O , a duplicate detection technique that operates purely on video-based bug reports by leveraging both visual and textual information. TANGO combines tailored computer vision techniques, optical character recognition, and text retrieval. We evaluated multiple configurations of Ta n g o in a comprehensive empirical evaluation on 4,860 duplicate detection tasks that involved a total of 180 screenrecordings from six Android apps. Additionally, we conducted a user study investigating the effort required for developers to manually detect duplicate video-based bug reports and compared this to the effort required to use TA N G O . The results reveal that TA N G O 's optimal configuration is highly effective at detecting duplicate video-based bug reports, accurately ranking target duplicate videos in the top-2 returned results in 83% of the tasks. Additionally, our user study shows that, on average, TANGO can reduce developer effort by over 60%, illustrating its practicality.

show abstract

Section: B Duplicate Detection Tasksmentioning

confidence: 99%

Section: Tango's Execution and Effectiveness Measurementmentioning

confidence: 99%