How are functionally similar code clones syntactically different? An empirical study and a benchmark

Abstract

Background. Today, redundancy in source code, so-called ''clones'' caused by copy &paste can be found reliably using clone detection tools. Redundancy can arise also independently, however, not caused by copy&paste. At present, it is not clear how only functionally similar clones (FSC) differ from clones created by copy&paste. Our aim is to understand and categorise the syntactical differences in FSCs that distinguish them from copy&paste clones in a way that helps clone detection research. Methods. We conducted an experiment using known functionally similar programs in Java and C from coding contests. We analysed syntactic similarity with traditional detection tools and explored whether concolic clone detection can go beyond syntax. We ran all tools on 2,800 programs and manually categorised the differences in a random sample of 70 program pairs. Results. We found no FSCs where complete files were syntactically similar. We could detect a syntactic similarity in a part of the files in <16% of the program pairs. Concolic detection found 1 of the FSCs. The differences between program pairs were in the categories algorithm, data structure, OO design, I/O and libraries. We selected 58 pairs for an openly accessible benchmark representing these categories. Discussion. The majority of differences between functionally similar clones are beyond the capabilities of current clone detection approaches. Yet, our benchmark can help to drive further clone detection research.

Improving Syntactical Clone Detection Methods through the Use of an Intermediate Representation

“…We sampled 10 solutions from 30 different problems for a total of 44,850 4 function pairs. Since these solutions all passed the automated test suite from Google they can be considered as type-4 clones [18], [19]. Out of all possible pairs, 1,350 are clones (solutions which were submitted to the same problem).…”

“…• Tools built with type-1, and -2 in mind use syntactic [5] or lexical similarities [3] to detect clones. By definition, these methods cannot detect semantic similarities if the syntax used to implement them is different [19]. Their performance for type-4 clones is thus lackluster.…”

Towards Semantic Clone Detection via Probabilistic Software Modeling

“…PDG based methods can detect complex Type 3 clones, e.g., Listings 1 and 2. However, the compared PDG sub-graphs are a representation of the source code; thereby, the approaches still rely on syntactic similarity [43].…”

Are there functionally similar code clones in practice?

“…Finally, Wagner et al found that less than 16 % of FSC pairs have actual syntactic similarities [3]. They provide a benchmark for FSCs which has, to the best of our knowledge, not yet been used to test FSC detection approaches.…”

“…The problem with classic approaches based on text, tokens, or syntax is that they cannot find clones with a completely different structure. We include those in the so called Functionally Similar Clones (FSCs) [3]. FSCs have the same or similar functionality but were generally created independently.…”