FAST2: An intelligent assistant for finding relevant papers

Yu, Zhe

doi:10.1016/j.eswa.2018.11.021

Cited by 63 publications

(110 citation statements)

References 44 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…Our pre-experimental belief was that EMBLEM would require extensive tuning before it could be used for labelling Github commits. However, the effectiveness of EMBLEM was obtained using Yu et al's original decisions [114], [116] without extensive tuning. Future improvements can be achieved by tuning different settings.…”

Section: Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Standard automatic methods for recognizing problematic development commits can be greatly improved via the incremental application of human+artificial expertise. In this approach, called EMBLEM, an AI tool first explore the software development process to label commits that are most problematic. Humans then apply their expertise to check those labels (perhaps resulting in the AI updating the support vectors within their SVM learner). We recommend this human+AI partnership, for several reasons. When a new domain is encountered, EMBLEM can learn better ways to label which comments refer to real problems. Also, in studies with 9 open source software projects, labelling via EMBLEM's incremental application of human+AI is at least an order of magnitude cheaper than existing methods (≈ eight times). Further, EMBLEM is very effective. For the data sets explored here, EMBLEM better labelling methods significantly improved Popt20 and G-score performance in nearly all the projects studied here. TABLE 1This paper argues against using keywords like these as a method for labelling a commit as "buggy'.

show abstract

Section: Frameworkmentioning

confidence: 99%

“…Also, we could explore other control parameters for EMBLEM. All the above results were obtained using Yu et al's [114], [116] original requirements (e.g. the values of {N 1 = 4000, N 2 = 1, N 3 = 30, N 4 = 95%}) within the EMBLEM method.…”

Section: Future Workmentioning

confidence: 99%

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In practice, active learning-based solutions of total recall problems focus on the following three targets [41], [32], [31], [13]: • Target 1 efficiency: achieving higher recall with a lower cost than other solutions. This is the main target of total recall problems as defined in §3.1.…”

Section: Active Learning-based Frameworkmentioning

confidence: 99%

Improving Vulnerability Inspection Efficiency Using Active Learning

Theisen

Williams

2021

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Software engineers can find vulnerabilities with less effort if they are directed towards code that might contain more vulnerabilities. HARMLESS is an incremental support vector machine tool that builds a vulnerability prediction model from the source code inspected to date, then suggests what source code files should be inspected next. In this way, HARMLESS can reduce the time and effort required to achieve some desired level of recall for finding vulnerabilities. The tool also provides feedback on when to stop (at that desired level of recall) while at the same time, correcting human errors by double-checking suspicious files.This paper evaluates HARMLESS on Mozilla Firefox vulnerability data. HARMLESS found 80, 90, 95, 99% of the vulnerabilities by inspecting 10, 16, 20, 34% of the source code files. When targeting 90, 95, 99% recall, HARMLESS could stop after inspecting 23, 30, 47% of the source code files. Even when human reviewers fail to identify half of the vulnerabilities (50% false negative rate), HARMLESS could detect 96% of the missing vulnerabilities by double-checking half of the inspected files.Our results serve to highlight the very steep cost of protecting software from vulnerabilities (in our case study that cost is, for example, the human effort of inspecting 28,750 × 20% = 5,750 source code files to identify 95% of the vulnerabilities). While this result could benefit the mission-critical projects where human resources are available for inspecting thousands of source code files, the research challenge for future work is how to further reduce that cost. The conclusion of this paper discusses various ways that goal might be achieved.

show abstract

“…Given the information available in automated UI testing, we extract three types of features: • Text feature: the same text mining feature extraction used in the total recall approaches [56,57] Using the foregoing types of features, the proposed framework is described in Algorithm 1 with engineering choices of N 1 , N 2 . N 1 is the batch size of the process.…”

Section: Terminatormentioning

confidence: 99%

“…N 2 represents the threshold above which certainty sampling is applied instead of uncertainty sampling. In this paper, we chose N 2 = 30 as suggested by previous works on total recall [57].…”

Section: Terminatormentioning

confidence: 99%

TERMINATOR: better automated UI test case prioritization

Fahid

Rothermel

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

Self Cite

View full text Add to dashboard Cite

Automated UI testing is an important component of the continuous integration process of software development. A modern web-based UI is an amalgam of reports from dozens of microservices written by multiple teams. Queries on a page that opens up another will fail if any of that page's microservices fails. As a result, the overall cost for automated UI testing is high since the UI elements cannot be tested in isolation. For example, the entire automated UI testing suite at LexisNexis takes around 30 hours (3-5 hours on the cloud) to execute, which slows down the continuous integration process.To mitigate this problem and give developers faster feedback on their code, test case prioritization techniques are used to reorder the automated UI test cases so that more failures can be detected earlier. Given that much of the automated UI testing is "black box" in nature, very little information (only the test case descriptions and testing results) can be utilized to prioritize these automated UI test cases. Hence, this paper evaluates 17 "black box" test case prioritization approaches that do not rely on source code information. Among these, we propose a novel TCP approach, that dynamically re-prioritizes the test cases when new failures are detected, by applying and adapting a state of the art framework from the total recall problem. Experimental results on LexisNexis automated UI testing data show that our new approach (which we call TERMINATOR), outperformed prior state of the art approaches in terms of failure detection rates with negligible CPU overhead. CCS CONCEPTS• Software and its engineering → Software testing and debugging; • Information systems → Learning to rank.

show abstract

FAST2: An intelligent assistant for finding relevant papers

Cited by 63 publications

References 44 publications

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Better Data Labelling With EMBLEM (and how that Impacts Defect Prediction)

Improving Vulnerability Inspection Efficiency Using Active Learning

TERMINATOR: better automated UI test case prioritization

Contact Info

Product

Resources

About