System Testing of Timing Requirements Based on Use Cases and Timed Automata

Wang, Chunhui; Pastore, Fabrizio; Briand, Lionel C.

doi:10.1109/icst.2017.34

Cited by 9 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The test oracle process in ReNaLART does not require the conversion of OCL constraints and thus results in reduced time. Wang, Pastore, and Briand (2017) propose another approach for system testing in which test cases are generated automatically from use case specifications. The approach generates executable test cases from natural language requirements in combination with timed automata which is again a type of formalism.…”

Section: Test Case Generation From Requirementsmentioning

confidence: 99%

Automating test oracles from restricted natural language agile requirements

et al. 2020

View full text Add to dashboard Cite

Manual testing of software requirements written in natural language for agile or any other methodology requires more time and human resources. This leaves the testing process error prone and time consuming. For satisfied end users with bug-free software delivered on time, there is a need to automate the test oracle process for natural language or informal requirements. The automation of the test oracle is relatively easier with formal requirements, but this task is difficult to achieve with natural language requirements. This study proposes an approach called Restricted Natural Language Agile Requirements Testing (ReNaLART) to automate the test oracle from restricted natural language agile requirements. For this purpose, it uses an existing user story template with some modifications for writing user stories. This helps in identifying test input and expected output for a user story. For comparison of expected and observed outputs it makes use of a regex pattern and string distance functions. It is capable of assigning different types of verdicts automatically depending upon the similarity/ dissimilarity between observed and expected outputs of user stories. ReNaLART is validated using several case studies of different domains, namely, OLX Pakistan, Mental Health Tests, McDelivery Pakistan, BlueStacks, Power Searching with Google, TensorFlow Playground, w3Schools 2018 offline and Touch'D. It revealed several faults in five of the above listed eight applications. Plus, the proposed test oracle on an average took 0.02 s for test data generation, expected output generation and verdict assignment. Both these facts show the fault revealing effectiveness and efficiency of ReNaLART. K E Y W O R D S automated test oracle, automated test-driven development, behaviour-driven development, restricted natural language, agile requirements 1 | INTRODUCTION Starting from the classic waterfall process model to the more recent agile process models, the focus of development has shifted from clearly gathered and well-documented requirements to handle creeping requirements with less focus on spending ample time on documenting requirements. Since the agile process caters for changing requirements, this poses a challenge for testers to test functionality corresponding to these changing requirements in an automated and efficient manner. As natural language is used widely as a tool in the software industry for documenting

show abstract

Section: Test Case Generation From Requirementsmentioning

confidence: 99%

Automating test oracles from restricted natural language agile requirements

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In [SP35], the authors demonstrate that Use Cases can be employed in Restricted Natural Language Text; facilitating the derivation of test cases. Use Case specifications can be used to identify the test inputs that fire the state transitions in timed automata [SP21]. Effectiveness of Use Cases in test effort estimation is explored by [SP28].…”

Section: 1mentioning

confidence: 99%

The impact of Use Cases in real-world software development projects: A systematic mapping study

Barros-Justo

Benitti

Tiwari

2019

Computer Standards & Interfaces

View full text Add to dashboard Cite

Context: There is abundant literature on the application of UML Use Cases. However, the impact that these applications have had on real projects (in industry) is not well known. It is necessary to know what the impact of the Use Cases really is, so that both, researchers and professionals can make the most pertinent decisions.Objective: To identify and classify the positive and negative impacts of using Use Cases in real-world settings. Method:We conducted a systematic mapping study. The search strategies retrieved a set of 4,431 papers out of which 47 were selected as primary studies. We defined four facets to classify these studies: 1) the positive impact (advantages), 2) the negative impact (disadvantages), 3) the industry's domain and 4) the type of research reported. Results:Our study identified eight categories of advantages related to the application of Use Cases. The most mentioned were estimation, analysis and automation. These advantages had a heterogeneous distribution along the years. On the other hand, the granularity of the scenarios described in the Use Cases, the lack of a standardized format for specifying requirements, and the lack of appropriate guidelines for analysing them were the most mentioned disadvantages. We identified a variety of industry domains, grouped into seven categories. As we can expect most of the papers report experiences in the Information Technology domain, followed by financials applications. Almost half the papers applied evaluation research, including an empirical validation. Only one third of the analysed papers reported threats to validity, the most mentioned being generalizability (38%). Conclusions:Use Cases have proven to be a useful tool in software development, particularly during the early stages. The positive effects far outweigh the few negative effects reported, although this may be due to the researchers' tendency of not reporting negative results.

show abstract

“…This limited observability makes oracles that simply check if the expected output sequence has been generated by test inputs ineffective. More precisely, these oracles may not detect transition faults [29,39], which manifest themselves when the active location of a TA following a state transition is incorrect but the system updates its state variables and generates outputs as expected.…”

Section: Testing With Uio Sequencesmentioning

confidence: 99%

“…To perform the experiment, and avoid bias in the selection of the test cases, for the case of BodySense T M we considered ten test suites of 122 test cases each, which were automatically generated in previous work for the same software under test [39]. The test suites were generated using the TAUC test generation technique (detailed in [39]), which is a model-based testing technique that relies on meta-heuristic search to generate test suites that (1) cover all the transitions in a network of TAs and (2) stress timing requirements by sending multiple, diverse inputs to the system. TAUC maximizes the diversity among the sequences of inputs being sent to the system by the different test cases in a test suite.…”

Section: Rq2mentioning

confidence: 99%

“…TAUC maximizes the diversity among the sequences of inputs being sent to the system by the different test cases in a test suite. In previous work, TAUC has shown to be able to generate test suites that discover faults not detected by the test suite manually produced by software engineers [39]. For the case of HOD, we considered 100 test cases generated according to the same criterion adopted for the BodySense T M test suites.…”

Section: Rq2mentioning

confidence: 99%

See 1 more Smart Citation

Oracles for Testing Software Timeliness with Uncertainty

Wang

Pastore

Briand

2018

ACM Trans. Softw. Eng. Methodol.

Self Cite

View full text Add to dashboard Cite

Uncertainty in timing properties (e.g., detection time of external events) is a common occurrence in embedded software systems since these systems interact with complex physical environments. Such time uncertainty leads to non-determinism. For example, time-triggered operations may either generate different valid outputs across different executions, or experience failures (e.g., results not being generated in the expected time window) that occur only occasionally over many executions. For these reasons, time uncertainty makes the generation of effective test oracles for timing requirements a challenging task. To address the above challenge, we propose STUIOS (Stochastic Testing with Unique Input Output Sequences), an approach for the automated generation of stochastic oracles that verify the capability of a software system to fulfill timing constraints in the presence of time uncertainty. Such stochastic oracles entail the statistical analysis of repeated test case executions based on test output probabilities predicted by means of statistical model checking. Results from two industrial case studies in the automotive domain demonstrate that this approach improves the fault detection effectiveness of tests suites derived from timed automata, compared to traditional approaches. CCS Concepts: • Software and its engineering → Software verification and validation;

show abstract

System Testing of Timing Requirements Based on Use Cases and Timed Automata

Cited by 9 publications

References 26 publications

Automating test oracles from restricted natural language agile requirements

Automating test oracles from restricted natural language agile requirements

The impact of Use Cases in real-world software development projects: A systematic mapping study

Oracles for Testing Software Timeliness with Uncertainty

Contact Info

Product

Resources

About