CP-Miner: finding copy-paste and related bugs in large-scale software code

Li, Z.; Lu, Shan; Myagmar, Suvda; Zhou, Yuanyuan

doi:10.1109/tse.2006.28

Cited by 608 publications

(513 citation statements)

References 25 publications

Supporting

Mentioning

490

Contrasting

Unclassified

Order By: Relevance

“…Different similarity measurements such as suffix trees, string alignment, Jaccard similarity, etc., can be applied to sequences or sets of tokens. Tools that rely on tokens include Sherlock (Joy and Luck 1999), BOSS (Joy et al 2005), Sim (Gitchell and Tran 1999), YAP3 (Wise 1996), JPlag (Prechelt et al 2002), CCFinder (Kamiya et al 2002), CP-Miner (Li et al 2006), MOSS (Schleimer et al 2003), Burrows et al (2007), and the Source Code Similarity Detector System (SCSDS) (Duric and Gasevic 2013). The token-based representation is widely used in source code similarity measurement and very efficient on a scale of millions SLOC.…”

Section: Code Similarity Measurementmentioning

confidence: 99%

“…These techniques show positive results and open further possibilities in this research area. Examples of these techniques include Software Bertillonage (Davies et al 2013), Kolmogorov complexity (Li and Vitâanyi 2008), Latent Semantic Indexing (LSI) (McMillan et al 2012), and Latent Semantic Analysis (LSA) (Cosma and Joy 2012).…”

Section: Code Similarity Measurementmentioning

confidence: 99%

“…It is an approximation of the normalised information distance which is in turn based on the concept of Kolmogorov complexity (Li and Vitâanyi 2008). The NCD between two documents can be computed by…”

Section: Compression Toolsmentioning

confidence: 99%

See 2 more Smart Citations

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

Copying and pasting of source code is a common activity in software engineering. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental scenarios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for three of the tools. Moreover, we demonstrate that optimal configurations are very sensitive to a specific data set. After directly applying optimal configurations derived from one data set to another, the tools perform poorly on the new data set. The code similarity analysers are thoroughly evaluated not only based on several well-known pair-based and query-based error measures but also on each specific type of pervasive code modification. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code.

show abstract

Section: Code Similarity Measurementmentioning

confidence: 99%

Section: Code Similarity Measurementmentioning

confidence: 99%

See 1 more Smart Citation

A comparison of code similarity analysers

2017

View full text Add to dashboard Cite

show abstract

“…Many approaches have been developed over the years to detect code clones [20,23,25,26]. A code clone is two or more segments of code that have the same semantics but come from different sources.…”

Section: Code Clones and Reuse Detectionmentioning

confidence: 99%

AnDarwin: Scalable Detection of Semantically Similar Android Applications

Crussell

Gibler

Chen

2013

Lecture Notes in Computer Science

112

View full text Add to dashboard Cite

Abstract. The popularity and utility of smartphones rely on their vibrant application markets; however, plagiarism threatens the long-term health of these markets. We present a scalable approach to detecting similar Android apps based on their semantic information. We implement our approach in a tool called AnDarwin and evaluate it on 265,359 apps collected from 17 markets including Google Play and numerous thirdparty markets. In contrast to earlier approaches, AnDarwin has four advantages: it avoids comparing apps pairwise, thus greatly improving its scalability; it analyzes only the app code and does not rely on other information -such as the app's market, signature, or descriptionthus greatly increasing its reliability; it can detect both full and partial app similarity; and it can automatically detect library code and remove it from the similarity analysis. We present two use cases for AnDarwin: finding similar apps by different developers ("clones") and similar apps from the same developer ("rebranded"). In ten hours, AnDarwin detected at least 4,295 apps that have been the victims of cloning and 36,106 apps that are rebranded. By analyzing the clusters found by AnDarwin, we found 88 new variants of malware and identified 169 malicious apps based on differences in the requested permissions. Our evaluation demonstrates AnDarwin's ability to accurately detect similar apps on a large scale.

show abstract

“…Different analysis tasks pay attention to different application aspects, such as system failure tracing [19,10], event correlation discovery [20,18,16], and event based trend analysis [5,6,7]. In practice, these methods are often conducted when the analysts already have some prior knowledge about the data.…”

Section: Related Workmentioning

confidence: 99%

META: Multi-resolution Framework for Event Summarization

Jiang

Perng

2014

Proceedings of the 2014 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Event summarization is an effective process that mines and organizes event patterns to represent the original events. It allows the analysts to quickly gain the general idea of the events. In recent years, several event summarization algorithms have been proposed, but they all focus on how to find out the optimal summarization results, and are designed for one-time analysis. As event summarization is a comprehensive analysis work, merely handling this problem with a single optimal algorithm is not enough.In the absence of an integrated summarization solution, we propose an extensible framework -META -to enable analysts to easily and selectively extract and summarize events from different views with different resolutions. In this framework, we store the original events in a carefully-designed data structure that enables an efficient storage and multiresolution analysis. On top of the data model, we define a summarization language that includes a set of atomic operators to manipulate the meta-data. Furthermore, we present 5 commonly used summarization tasks, and show that all these tasks can be easily expressed by the language. Experimental evaluation on both real and synthetic datasets demonstrates the efficiency and effectiveness of our framework.

show abstract

CP-Miner: finding copy-paste and related bugs in large-scale software code

Cited by 608 publications

References 25 publications

A comparison of code similarity analysers

A comparison of code similarity analysers

AnDarwin: Scalable Detection of Semantically Similar Android Applications

META: Multi-resolution Framework for Event Summarization

Contact Info

Product

Resources

About