Context: Defect prediction research is based on a small number of defect datasets and most are at class not method level. Consequently our knowledge of defects is limited. Identifying defect datasets for prediction is not easy and extracting quality data from identified datasets is even more difficult. Goal: Identify open source Java systems suitable for defect prediction and extract high quality fault data from these datasets. Method: We used the Boa to identify candidate open source systems. We reduce 50,000 potential candidates down to 23 suitable for defect prediction using a selection criteria based on the system's software repository and its defect tracking system. We use an enhanced SZZ algorithm to extract fault information and calculate metrics using JHawk. Result: We have produced 138 fault and metrics datasets for the 23 identified systems. We make these datasets (the ELFF datasets) and our data extraction tools freely available to future researchers. Conclusions: The data we provide enables future studies to proceed with minimal effort. Our datasets significantly increase the pool of systems currently being used in defect analysis studies.

Background: Test quality is a prerequisite for achieving production system quality. While the concept of quality is multidimensional, most of the effort in testing context has been channeled towards measuring test effectiveness. Objective: While effectiveness of tests is certainly important, we aim to identify a core list of testing principles that also address other quality facets of testing, and to discuss how they can be quantified as indicators of test quality. Method: We have conducted a two-day workshop with our industry partners to come up with a list of relevant principles and best practices expected to result in high quality tests. We then utilised our academic and industrial training materials together with recommendations in practitioner oriented testing books to refine the list. We surveyed existing literature for potential metrics to quantify identified principles. Results: We have identified a list of 15 testing principles to capture the essence of testing goals and best practices from quality perspective. Eight principles do not map to existing test smells and we propose metrics for six of those. Further, we have identified additional potential metrics for the seven principles that partially map to test smells. Conclusion: We provide a core list of testing principles along with a discussion of possible ways to quantify them for assessing goodness of tests. We believe that our work would be useful for practitioners in assessing the quality of their tests from multiple perspectives including but not limited to maintainability, comprehension and simplicity.

Context: Identifying defects in code early is important. A wide range of static code metrics have been evaluated as potential defect indicators. Most of these metrics offer only high level insights and focus on particular pre-selected features of the code. None of the currently used metrics clearly performs best in defect prediction. Objective: We use Abstract Syntax Tree (AST) n-grams to identify features of defective Java code that improve defect prediction performance. Method: Our approach is bottom-up and does not rely on pre-selecting any specific features of code. We use non-parametric testing to determine relationships between AST n-grams and faults in both open source and commercial systems. We build defect prediction models using three machine learning techniques. Results: We show that AST n-grams are very significantly related to faults in some systems, with very large effect sizes. The occurrence of some frequently occurring AST n-grams in a method can mean that the method is up to three times more likely to contain a fault. AST n-grams can have a large effect on the performance of defect prediction models. Conclusions: We suggest that AST n-grams offer developers a promising approach to identifying potentially defective code.

Defect prediction has been the subject of a great deal of research over the last two decades. Despite this research it is increasingly clear that defect prediction has not transferred into industrial practice. One of the reasons defect prediction remains a largely academic activity is that there are no defect prediction tools that developers can use during their day-today development activities. In this paper we describe the defect prediction tool that we have developed for industrial use. Our ELFF tool seamlessly plugs into the IntelliJ IDE and enables developers to perform regular defect prediction on their Java code. We explain the state-of-art defect prediction that is encapsulated within the ELFF tool and describe our evaluation of ELFF in a large UK telecommunications company.

Background: Software Code Cloning is widely used by developers to produce code in which they have confidence and which reduces development costs and improves the software quality. However, Fowler and Beck suggest that the maintenance of clones may lead to defects and therefore clones should be re-factored out. Objective: We investigate the purpose of code cloning, the detection techniques developed and the datasets used in software code cloning studies between the years of 2007 and 2011. This is to analyse the current research trends in code cloning to try and find techniques which have been successful in identifying clones used for defect prediction. Method: We used a mapping study to identify 220 software code cloning studies published from January 2007 to December 2011. We use these papers to answer six research questions by analysing their abstracts, titles and reading the papers themselves. Results: The main focus of studies is the technique of software code clone detection. In the past four years the number of studies being accepted at conferences and in journals has risen by 71%. Most datasets are only used once, therefore the performance reported by one paper is not comparable with the performance reported by another study. Conclusion: The techniques used to detect clones seem to be the main focus of studies. However it is difficult to compare the performance of the detection tools reported in different studies because the same dataset is rarely used in more than one paper. There are few benchmark datasets where the clones have been correctly identified. Few studies apply code cloning detection to defect prediction