Lessons learned from automating tests for an operations support system

Fecko, Mariusz A.; Lott, Christopher M.

doi:10.1002/spe.491

Cited by 5 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The high complexity of a test automation system is a risk to the perceived value of the automation. If manual testing is seen as easier to perform than writing the automated test scripts , the consequence may be that testers choose to work manually instead of automating tests, in particular if there is time pressure .…”

Section: Resultsmentioning

confidence: 99%

“…Inadequate development practices A critical insight is that test automation is software development and need to be treated as such . This means that there is a clear need for staff skilled in programming and an understanding of how to do software development , to be able to do effective and efficient development of the test automation.…”

Section: Resultsmentioning

confidence: 99%

“…Resolving configuration and setup issues may consume a large part of the time and resources dedicated to testing, to a point where the automation can be considered to be useless .…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Impediments for software test automation: A systematic literature review

Wiklund

Eldh

Sundmark

et al. 2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

SummaryAutomated software testing is a critical enabler for modern software development, where rapid feedback on the product quality is expected. To make the testing work well, it is of high importance that impediments related to test automation are prevented and removed quickly. An enabling factor for all types of improvement is to understand the nature of what is to be improved. We have performed a systematic literature review of reported impediments related to software test automation to contribute to this understanding. In this paper, we present the results from the systematic literature review: The list of identified publications, a categorization of identified impediments, and a qualitative discussion of the impediments proposing a socio-technical system model of the use and implementation of test automation. 30% to 80% of the development costs are reported to be related to testing [2,3] . The high use of test automation also makes the automated tests mission critical: If the test automation stops working or slows down, the development will also stop or slow down.The investment cost for test automation can be significant. It is difficult to propose a generalizable cost model, due to a lack of published information available on the cost of ownership for test automation. However, the size and complexity of a test system can be in the order of, or often larger than, the complexity and size of the tested product [10,11][IP7] [12]. This makes it reasonable to conjecture that the cost of ownership of an automated test execution system is in the order of the cost of development of the product it is testing. Even if a lot less effort is spent on the test systems than on the sold products, it still represents a major investment for the software business as a whole. If the test automation does not perform as well as expected, this investment is unlikely to be recovered as planned, which may have significant consequences for the business. Further, it is reasonable to conjecture that test systems suffer from similar technical debt issues as regular systems do. Technical debt is a term that is used to describe the gap between the current state of the system and the ideal state of the system [13], usually in situations where a compromise is made during development to meet demands in one dimension, such as lead time, by sacrificing work in another dimension, such as architecture.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Impediments for software test automation: A systematic literature review

Wiklund

Eldh

Sundmark

et al. 2017

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…Gut check: Also "data gut check." Quick, broad, and shallow testing (44) before and during data analysis. Synonymous words: smoke test, sanity check (45) , consistency check, sniff test, soundness check.…”

Section: Boxes Box 1: Terminologymentioning

confidence: 99%

“…Gut check: Also “data gut check.” Quick, broad, and shallow testing [ 48 ] before and during data analysis. Although this is usually described in the context of software development, the concept of a data-specific gut check can include checking the dimensions of data structures after merging or assessing null values/missing values, zero values, negative values, and ranges of values to see if they make sense (synonymous words: smoke test, sanity check [ 49 ], consistency check, sniff test, soundness check).…”

Section: Introductionmentioning

confidence: 99%

Principles for data analysis workflows

2021

View full text Add to dashboard Cite

A systematic and reproducible “workflow”—the process that moves a scientific investigation from raw data to coherent research question to insightful contribution—should be a fundamental part of academic data-intensive research practice. In this paper, we elaborate basic principles of a reproducible data analysis workflow by defining 3 phases: the Explore, Refine, and Produce Phases. Each phase is roughly centered around the audience to whom research decisions, methodologies, and results are being immediately communicated. Importantly, each phase can also give rise to a number of research products beyond traditional academic publications. Where relevant, we draw analogies between design principles and established practice in software development. The guidance provided here is not intended to be a strict rulebook; rather, the suggestions for practices and tools to advance reproducible, sound data-intensive analysis may furnish support for both students new to research and current researchers who are new to data-intensive work.

show abstract