Threshold-free code clone detection for a large-scale heterogeneous Java repository

Keivanloo, Iman; Zhang, Feng; Zou, Ying

doi:10.1109/saner.2015.7081830

Cited by 26 publications

(13 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An empirical study of using compilation/decompilation to enhance the performance of clone detection tool in three real-world system found similar results to our study (Ragkhitwetsagul and Krinke 2017b). Keivanloo et al (2015) discussed the problem of using a single threshold for clone detection over several repositories and propose a solution using threshold-free clone detection based on unsupervised learning. The method mainly utilises k-means clustering with the Friedman quality optimisation method.…”

Section: Related Worksupporting

confidence: 81%

“…The configuration problem for clone detection tools including setting thresholds has been mentioned by several studies as one of the threats to validity (Wang et al 2001). There has also been an initiative to avoid using thresholds at all for clone detection (Keivanloo et al 2015). Hence, we try to avoid the problem of threshold sensitivity affecting our results.…”

Section: Scenario 4 (Ranked Results)mentioning

confidence: 99%

See 1 more Smart Citation

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

Copying and pasting of source code is a common activity in software engineering. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental scenarios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for three of the tools. Moreover, we demonstrate that optimal configurations are very sensitive to a specific data set. After directly applying optimal configurations derived from one data set to another, the tools perform poorly on the new data set. The code similarity analysers are thoroughly evaluated not only based on several well-known pair-based and query-based error measures but also on each specific type of pervasive code modification. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code.

show abstract

Section: Related Worksupporting

confidence: 81%

Section: Scenario 4 (Ranked Results)mentioning

confidence: 99%

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

show abstract

“…The code clone pairs were then manually validated for the training phase of the proposed method. As some recent research shows that the clone validation decision in some scenario depends on user's perspective [25], that is given a possible code clone pair to validate some judges might decide it to be a true positive clone pairs where others might say the opposite (especially in case of Type 3 and Type 4 clones). So to consider this generalization to the proposed method the whole set of code pairs were split into five parts to be validated by five different graduate research students from computer science background.…”

Section: High-level Details Of the Data Setmentioning

confidence: 99%

CloneCognition: machine learning based code clone validation tool

Mostaeen

Svajlenko

Roy

et al. 2019

Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of

View full text Add to dashboard Cite

A code clone is a pair of code fragments, within or between software systems that are similar. Since code clones often negatively impact the maintainability of a software system, several code clone detection techniques and tools have been proposed and studied over the last decade. However, the clone detection tools are not always perfect and their clone detection reports often contain a number of false positives or irrelevant clones from specific project management or user perspective. To detect all possible similar source code patterns in general, the clone detection tools work on the syntax level while lacking user-specific preferences. This often means the clones must be manually inspected before analysis in order to remove those false positives from consideration. This manual clone validation effort is very time-consuming and often error-prone, in particular for large-scale clone detection. In this paper, we propose a machine learning approach for automating the validation process. First, a training dataset is built by taking code clones from several clone detection tools for different subject systems and then manually validating those clones. Second, several features are extracted from those clones to train the machine learning model by the proposed approach. The trained algorithm is then used to automatically validate clones without human inspection. Thus the proposed approach can be used to remove the false positive clones from the detection results, automatically evaluate the precision of any clone detectors for any given set of datasets, evaluate existing clone benchmark datasets, or even be used to build new clone benchmarks and datasets with minimum

show abstract

“…However, it is not clear if duplicate logging statements are indeed associated with code clones. Moreover, the performance of clone detection tools is dependent on the thresholds [54,34,50,40]. Choosing the optimal thresholds is a non-trivial task and the value may differ across systems [62].…”

Section: Construct Validitymentioning

confidence: 99%

Characterizing and Detecting Duplicate Logging Code Smells

2019

2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)

View full text Add to dashboard Cite

School of Graduate StudiesThis is to certify that the thesis prepared By: Zhenhao Li Entitled: Characterizing and Detecting Duplicate Logging Code Smells and submitted in partial fulfillment of the requirements for the degree of Master of Applied Science (Software Engineering) complies with the regulations of this University and meets the accepted standards with respect to originality and quality. Abstract Characterizing and Detecting Duplicate Logging Code Smells Zhenhao Li Developers rely on software logs for a wide variety of tasks, such as debugging, testing, program comprehension, verification, and performance analysis. Despite the importance of logs, prior studies show that there is no industrial standard on how to write logging statements. Recent research on logs often only considers the appropriateness of a log as an individual item (e.g., one single logging statement); while logs are typically analyzed in tandem. In this thesis, we focus on studying duplicate logging statements, which are logging statements that have the same static text message. Such duplications in the text message are potential indications of logging code smells, which may affect developers' understanding of the dynamic view of the system. We manually studied over 3K duplicate logging statements and their surrounding code in four large-scale open source systems:Hadoop, CloudStack, ElasticSearch, and Cassandra. We uncovered five patterns of duplicate logging code smells. For each instance of the code smell, we further manually identify the problematic (i.e., require fixes) and justifiable (i.e., do not require fixes) cases. Then, we contact developers in order to verify our manual study result. We integrated our manual study result and developers' feedback into our automated static analysis tool, DLFinder, which automatically detects problematic duplicate logging code smells. We evaluated DLFinder on the four manually studied systems and four additional systems: Kafka, Flink, Camel and Wicket. In total, combining the results of DLFinder and our manual analysis, we reported 91 problematic code smell instances to developers and all of them have been fixed. This thesis provides an initial step on creating a logging guideline for developers to improve the quality of logging code. DLFinder is also able to detect duplicate logging code smells with high precision and recall.iii

show abstract

Threshold-free code clone detection for a large-scale heterogeneous Java repository

Cited by 26 publications

References 35 publications

A comparison of code similarity analysers

A comparison of code similarity analysers

CloneCognition: machine learning based code clone validation tool

Characterizing and Detecting Duplicate Logging Code Smells

Contact Info

Product

Resources

About