We present ongoing work on HipSpec, a system for automatically deriving and proving properties about functional programs. HipSpec uses a combination of theory exploration, counter-example testing and inductive theorem proving to automatically generate a set of equational theorems about recursive functions in a program, which are later used as a background theory for proving stated properties about a program. Initial experiments are encouraging; our initial HipSpec prototype already compares favourably to other, similar systems.

QuickCheck allows us to verify software against particular properties. A property can be regarded as an abstraction over many unit tests. QuickCheck uses generated random input data to test such properties. If a counterexample is found, it becomes immediately clear what we have tested. This is not the case when all tests pass, since we do not (and shall not) see the actual generated test cases. How can we be sure about what is tested? QuickCheck has the ability to gather statistics about the test cases, which is insightful. But still it does not tell us whether the particular unit test scenarios we have in mind are included. For this reason, we have developed a tool that can answer this question. It checks if a given unit test can be generated by a property, making it easier to judge the property's quality. We have applied our tool to an industrial use case of testing the AUTOSAR basic software modules and shows that it can handle complex models and large unit tests.

We present an unbiased method for measuring the relative quality of different solutions to a programming problem. Our method is based on identifying possible bugs from program behaviour through black-box testing. The main motivation for such a method is its use in experimental evaluation of software development methods. We report on the use of our method in a small-scale such experiment, which was aimed at evaluating the effectiveness of property-based testing vs. unit testing in software development.

The testing of safety-related industrial systems is usually carried out by means of checklists. A tester has a list of scenarios that he or she manually applies to the system to check whether the system behaves according to its specification. However, operators behave unpredictably. Their behavior may not be covered by the set of scenarios tested and may lead to dangerous situations. To avoid this, randomized test case generation can be useful as it allows for unanticipated scenarios. The presented framework uses a tool for randomized test case generation, QuickCheck, to trigger event sequences that are then applied to a Safety Programmable Logic Controller (Safety PLC). Experiments show that this concept is capable of finding errors in safety code or increasing the tester's confidence in the correctness of the code by exhibiting a large number of passing test cases. While this concept proves to be powerful, it does not require much effort from the tester as the execution of test cases is done without user interaction.

No abstract