Navigating the maze: the impact of configurability in bioinformatics software

Cashman, Mikaela; Cohen, Myra B.; Ranjan, Priya; Cottingham, Robert W.

doi:10.1145/3238147.3240466

Cited by 26 publications

(22 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Halin et al demonstrated that uniform random sampling forms a strong baseline for faults and failure efficiency on the JHipster case [25]. We need additional cases with enumerable configuration spaces (like the BLAST model studied by Cashman et al [43]) to confirm or refute this result.…”

Section: Related Workmentioning

confidence: 95%

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Plazar

Acher

Perrouin

et al. 2019

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

Uniform or near-uniform generation of solutions for large satisfiability formulas is a problem of theoretical and practical interest for the testing community. Recent works proposed two algorithms (namely UniGen and QuickSampler) for reaching a good compromise between execution time and uniformity guarantees, with empirical evidence on SAT benchmarks. In the context of highly-configurable software systems (e.g., Linux), it is unclear whether UniGen and QuickSampler can scale and sample uniform software configurations. In this paper, we perform a thorough experiment on 128 real-world feature models. We find that UniGen is unable to produce SAT solutions out of such feature models. Furthermore, we show that QuickSampler does not generate uniform samples and that some features are either never part of the sample or too frequently present. Finally, using a case study, we characterize the impacts of these results on the ability to find bugs in a configurable system. Overall, our results suggest that we are not there: more research is needed to explore the cost-effectiveness of uniform sampling when testing large configurable systems.

show abstract

Section: Related Workmentioning

confidence: 95%

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Plazar

Acher

Perrouin

et al. 2019

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

show abstract

“…Both problems can be addressed, at least in part, by always reporting the exact method of counting: this affixes the meaning of standalone measurements, and allows the comparison of code with a loc measurement from a different paper. In our survey, only one paper [5] explains how loc was measured, and use a standard tool for this purpose.…”

Section: Why the Way We Actually Measure Code Is Badmentioning

confidence: 99%

“…Even if the result is code, it must be formatted to be measured in lines of code, but it already exists within the tool in a representation that can be measured more precisely, synthesis algorithm must be highly optimized to find the result. A different target program, baz (1,2,3,4,5,6) would also require 7 lines of code, but is only of height 1, and can be found very easily by even the most naive of algorithms.…”

Section: Case Study: Evaluating Projects That Produce Code As Outputmentioning

confidence: 99%

The wonderful wizard of LoC: paying attention to the man behind the curtain of lines-of-code metrics

Alpernas

Feldman

Peleg

2020

Proceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Sof

View full text Add to dashboard Cite

Lines-of-code metrics (loc) are commonly reported in Programming Languages (PL), Software Engineering (SE), and Systems papers. This convention has several different, often contradictory, goals, including demonstrating the 'hardness' of a problem, and demonstrating the 'easiness' of a problem. In many cases, the reporting of loc metrics is done not with a clearly communicated intention, but instead in an automatic, checkbox-ticking, manner.In this paper we investigate the uses of code metrics in PL, SE, and System papers. We consider the different goals that reporting metrics aims to achieve, several various domains wherein metrics are relevant, and various alternative metrics and their pros and cons for the different goals and domains. We argue that communicating claims about research software is usually best achieved not by reporting quantitative metrics, but by reporting the qualitative experience of researchers, and propose guidelines for the cases when quantitative metrics are appropriate.We end with a case study of the one area in which lines of code are not the default measurement-code produced by papers' solutions-and identify how measurements offered are used to support an explicit claim about the algorithm. Inspired by this positive example, we call for other cogent measures to be developed to support other claims authors wish to make. CCS Concepts: • General and reference → Metrics.

show abstract

“…Despite the many existing and upcoming applications of bioinformatics that demonstrate great potential for healthcare advancement, bioinformatics scientists themselves have yet, for the most part, to evolve along with the bioinformatics field in terms of acquiring refined skills for developing high-quality, maintainable software ( Lawlor & Sleator, 2021 ). In a review of two popular bio-sequence alignment software packages for bioinformatics, FAST and BLAST ( Aryal, 2019 ; Leimeister et al, 2014 ; Cashman et al, 2018 ) concluded that both tools would greatly benefit from improved flexibility, usability, and user-friendliness. Sanders & Kelly (2008) , Oulas et al (2019) and Lawlor & Sleator (2021) reinforced the notion that the complex bioinformatics discipline and its increasing depth and volume of applications necessitate the development of reliable and maintainable software for use and expansion by bioinformatics professionals.…”

Section: Introductionmentioning

confidence: 99%

Improving bioinformatics software quality through incorporation of software engineering practices

Noor¹

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Background Bioinformatics software is developed for collecting, analyzing, integrating, and interpreting life science datasets that are often enormous. Bioinformatics engineers often lack the software engineering skills necessary for developing robust, maintainable, reusable software. This study presents review and discussion of the findings and efforts made to improve the quality of bioinformatics software. Methodology A systematic review was conducted of related literature that identifies core software engineering concepts for improving bioinformatics software development: requirements gathering, documentation, testing, and integration. The findings are presented with the aim of illuminating trends within the research that could lead to viable solutions to the struggles faced by bioinformatics engineers when developing scientific software. Results The findings suggest that bioinformatics engineers could significantly benefit from the incorporation of software engineering principles into their development efforts. This leads to suggestion of both cultural changes within bioinformatics research communities as well as adoption of software engineering disciplines into the formal education of bioinformatics engineers. Open management of scientific bioinformatics development projects can result in improved software quality through collaboration amongst both bioinformatics engineers and software engineers. Conclusions While strides have been made both in identification and solution of issues of particular import to bioinformatics software development, there is still room for improvement in terms of shifts in both the formal education of bioinformatics engineers as well as the culture and approaches of managing scientific bioinformatics research and development efforts.

show abstract

Navigating the maze: the impact of configurability in bioinformatics software

Cited by 26 publications

References 33 publications

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

The wonderful wizard of LoC: paying attention to the man behind the curtain of lines-of-code metrics

Improving bioinformatics software quality through incorporation of software engineering practices

Contact Info

Product

Resources

About