With the emergence and spread of agile processes, the practices of writing and maintaining documentation have drastically changed in the last decade. In this work, we performed a qualitative study to explore the current practices for managing two related types of software documentation: requirements and acceptance tests. We interviewed twenty practitioners from seventeen business units in fifteen companies to investigate the companies' practices for writing, maintaining and linking requirements and acceptance test documentation. The study yields interesting and partially unexpected results. For example, we had expected that tests would be more extensively documented than requirements, while we found a strong linear correlation between the number of requirements and tests in our sample. We also found that technical people are usually not involved in the requirements engineering activities, which often results in misunderstood or underestimated requirements. Acceptance tests are written, in many cases, based on requirements that are not necessarily detailed enough. Also, acceptance tests are not regularly maintained, which occasionally results in confusing features and bugs. Abstract-With the emergence and spread of agile processes, the practices of writing and maintaining documentation have drastically changed in the last decade. In this work, we performed a qualitative study to explore the current practices for managing two related types of software documentation: requirements and acceptance tests. We interviewed twenty practitioners from seventeen business units in fifteen companies to investigate the companies' practices for writing, maintaining and linking requirements and acceptance test documentation. The study yields interesting and partially unexpected results. For example, we had expected that tests would be more extensively documented than requirements, while we found a strong linear correlation between the number of requirements and tests in our sample. We also found that technical people are usually not involved in the requirements engineering activities, which often results in misunderstood or underestimated requirements. Acceptance tests are written, in many cases, based on requirements that are not necessarily detailed enough. Also, acceptance tests are not regularly maintained, which occasionally results in confusing features and bugs.

Context and motivation] When a software-based system evolves, its requirements continuously change. This affects the acceptance tests, which must be adapted accordingly in order to maintain the quality of the evolving system. [Question/problem] In practice, requirements and acceptance test documents are not always aligned with each other, nor with the actual system behavior. Such inconsistencies may introduce software quality problems, unintended costs and project delays. [Principal ideas/results] To keep evolving requirements and their associated acceptance tests aligned, we are developing an approach called GuideGen that automatically generates guidance in natural language on how to modify impacted acceptance tests when a requirement is changed. We evaluated GuideGen using real-world data from three companies. For 262 non-trivial changes of requirements, we generated guidance on how to change the affected acceptance tests and evaluated the quality of this guidance with seven experts. The correctness of the guidance produced by our approach ranged between 67 and 89 percent of all changes for the three evaluated data sets. We further found that our approach performed better for agile requirements than for traditional ones. [Contribution] Our approach facilitates the alignment of acceptance tests with the actual requirements and also improves the communication between requirements engineers and testers.

Software requirements constantly change, thus impacting all other artifacts of an evolving system. In order to keep the system in a consistent state, changes in requirements should be documented and applied accordingly to all affected artifacts, including acceptance tests. In practice, however, changes in requirements are not always documented nor applied to the affected acceptance tests. This is mostly due to poor communication, lack of time or work overload, and eventually leads to project delays, unintended costs and unsatisfied customers. GuideGen is a tool-supported approach for keeping requirements and acceptance tests aligned. When a requirement is changed, GuideGen automatically generates guidance in natural language on how to modify impacted acceptance tests and communicates this information to the concerned parties. In this paper, we evaluate GuideGen in terms of its perceived usefulness for practitioners and its applicability to real software projects. The evaluation was conducted via interviews with 23 industrial practitioners from ten companies based in Europe. The results indicate that GuideGen is a useful approach that facilitates requirements change management and the communication of changes between requirements and test engineers. The participants also identified potential for improvement, in particular for using GuideGen in large projects.

No abstract

Keeping requirements and acceptance test documents aligned and up-to-date plays an important role in the success of software projects. In practice, these documents are not always aligned with each other, nor with the actual system behaviour. A previous study showed that even when requirements are updated, acceptance tests might stay outdated, which often leads to quality problems and unintended costs. In order to keep the requirements and test documents in a consistent state, we are developing an approach that automatically generates guidance on how to change impacted acceptance tests when changes in requirements occur. In this paper, we briefly present our approach and a prototype tool that implements it. A preliminary evaluation of our approach yielded encouraging results.