Siamese: scalable and incremental code clone search via multiple code representations

Abstract

This paper presents a novel code clone search technique that is accurate, incremental, and scalable to hundreds of million lines of code. Our technique incorporates multiple code representations (i.e., a technique to transform code into various representations to capture different types of clones), query reduction (i.e., a technique to select clone search keywords based on their uniqueness), and a customised ranking function (i.e., a technique to allow a specific clone type to be ranked on top of the search results) to improve clone search performance. We implemented the technique in a clone search tool, called Siamese, and evaluated its search accuracy and scalability on three established clone data sets. Siamese offers the highest mean average precision of 95% and 99% on two clone benchmarks compared to seven state-of-the-art clone detection tools, and reported the largest number of Type-3 clones compared to three other code search engines. Siamese is scalable and can return cloned code snippets within 8 seconds for a code corpus of 365 million lines of code. Using an index of 130,719 GitHub projects, we demonstrate that Siamese's incremental indexing capability dramatically decreases the index preparation time for large-scale data sets with multiple releases of software projects. The paper discusses the applications of Siamese to facilitate software development and research with two use cases including online code clone detection and clone search with automated license analysis.

Development of Cross Language Clone Detector for C, C & Java Repositories using Natural Language Processing

“…Many approaches exist to find the clones they are text-based [13], [14], token-based [15], [16], [17], [18]), tree-based [15]; [19], graph-based [16], or deep learning techniques [17] that match the similar codes among same project or different projects. Because of the large code repositories online, programmers never start coding from scratch which they obviously find it way to save time and avoid tedious work [1], [18], we can also find evident in the [18] to say that 70% of the code in GITHUB are clones.…”

“…Because of several MLOC's of code available on Internet, code search is becoming more common now a day's [1]. It is easier for developers to get code online than to start coding from scratch [1].…”

“…Because of several MLOC's of code available on Internet, code search is becoming more common now a day's [1]. It is easier for developers to get code online than to start coding from scratch [1]. But reusing code snippet from the online source adds maintenance cost and software quality [1] or may violate software licenses [1].…”

“…It is easier for developers to get code online than to start coding from scratch [1]. But reusing code snippet from the online source adds maintenance cost and software quality [1] or may violate software licenses [1]. Google and other text search engines cannot search source code directly, but can find similar code based on the text search.…”

“…Existing clone detection techniques can find clones of same language and very few recent studies show that they can find clones of same families. To help the managers and team leaders who involve in porting the code from one language to another, there needs a single code matching process that matches similar code clones in neighboring languages from large online or offline repositories [1].There are many approaches in finding the clones of similar languages [25], [26], [27], [28], [29]. With the invent of many mobile operating systems that provide similar features the popularity of multi language code clone match is on immediate need.…”