Toward Speeding up Mutation Analysis by Memoizing Expensive Methods

Ghanbari, Ali; Marcus, Andrian

doi:10.1109/icse-nier52604.2021.00023

Cited by 4 publications

(2 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use mutmut [9], a tool having over 600 stars on GitHub, to conduct mutation testing for Python solutions. Mutmut is widely adopted for mutation testing research on Python [15,22,24,71]. We use Mull [17,46] to conduct mutation testing for C++ solutions, which is also a widely used tool with more than 600 stars on GitHub.…”

Section: Mutation Score Analysismentioning

confidence: 99%

Who Judges the Judge: An Empirical Study on Online Judge Tests

Liu

Yan

Zhang

et al. 2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Online Judge platforms play a pivotal role in education, competitive programming, recruitment, career training, and large language model training. They rely on predefined test suites to judge the correctness of submitted solutions. It is therefore important that the solution judgement is reliable and free from potentially misleading false positives (i.e., incorrect solutions that are judged as correct). In this paper, we conduct an empirical study of 939 coding problems with 541,552 solutions, all of which are judged to be correct according to the test suites used by the platform, finding that 43.1% of the problems include false positive solutions (3,700 bugs are revealed in total). We also find that test suites are, nevertheless, of high quality according to widely-studied test effectiveness measurements: 88.2% of false positives have perfect (100%) line coverage, 78.9% have perfect branch coverage, and 32.5% have a perfect mutation score. Our findings indicate that more work is required to weed out false positive solutions and to further improve test suite effectiveness. We have released the detected false positive solutions and the generated test inputs to facilitate future research. CCS CONCEPTS• Software and its engineering → Software testing and debugging.

show abstract

Section: Mutation Score Analysismentioning

confidence: 99%

Who Judges the Judge: An Empirical Study on Online Judge Tests

Liu

Yan

Zhang

et al. 2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…Finally, we aim to match the potential benefits as closely as possible by pursuing all (correctness-preserving) memoization opportunities that arise during user app sessions. This is in contrast to prior memoization systems that either target testing environments with pre-determined execution patterns [35], or operate only on pure functions [90].…”

Section: Goals and Challengesmentioning

confidence: 99%

Floo

Ramanujam

Chen

Mardani

et al. 2022

Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services

View full text Add to dashboard Cite

Owing to growing feature sets and sluggish improvements to smartphone CPUs (relative to mobile networks), mobile app response times have increasingly become bottlenecked on client-side computations. In designing a solution to this emerging issue, our primary insight is that app computations exhibit substantial stability over time in that they are entirely performed in rarely-updated codebases within app binaries and the OS. Building on this, we present Floo, a system that aims to automatically reuse (or memoize) computation results during app operation in an effort to reduce the amount of compute needed to handle user interactions. To ensure practicality -the struggle with any memoization effort -in the face of limited mobile device resources and the short-lived nature of each app computation, Floo embeds several new techniques that collectively enable it to mask cache lookup overheads and ensure high cache hit rates, all the while guaranteeing correctness for any reused computations. Across a wide range of apps, live networks, phones, and interaction traces, Floo reduces median and 95th percentile interaction response times by 32.7% and 72.3%. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools; Mobile phones; Smartphones.

show abstract