Comparing techniques for authorship attribution of source code

Abstract

SUMMARYAttributing authorship of documents with unknown creators has been studied extensively for natural language text such as essays and literature, but less so for non‐natural languages such as computer source code. Previous attempts at attributing authorship of source code can be categorised by two attributes: the software features used for the classification, either strings of n tokens/bytes (n‐grams) or software metrics; and the classification technique that exploits those features, either information retrieval ranking or machine learning. The results of existing studies, however, are not directly comparable as all use different test beds and evaluation methodologies, making it difficult to assess which approach is superior. This paper summarises all previous techniques to source code authorship attribution, implements feature sets that are motivated by the literature, and applies information retrieval ranking methods or machine classifiers for each approach. Importantly, all approaches are tested on identical collections from varying programming languages and author types. Our conclusions are as follows: (i) ranking and machine classifier approaches are around 90% and 85% accurate, respectively, for a one‐in‐10 classification problem; (ii) the byte‐level n‐gram approach is best used with different parameters to those previously published; (iii) neural networks and support vector machines were found to be the most accurate machine classifiers of the eight evaluated; (iv) use of n‐gram features in combination with machine classifiers shows promise, but there are scalability problems that still must be overcome; and (v) approaches based on information retrieval techniques are currently more accurate than approaches based on machine learning. Copyright © 2012 John Wiley & Sons, Ltd.

An attempt toward Authorship Analysis of Obfuscated .NET Binaries

“…They believed executable code, even if optimized, still contains many features such as data structures and algorithms, compiler and system information and so on that may help to identify the author. In a recent survey study [6], previous attempts at attributing authorship of normal source code are categorized by two attributes: the source code metrics used for the classification, either strings of n tokens/bytes (n-grams) or object-oriented metrics such as number of classes, interfaces, etc. ; and the classification technique that exploits those features, either information retrieval ranking or machine learning.…”

Author Identification in Imbalanced Sets of Source Code Samples

“…Another proposed method concerns the extraction of token ngrams from code, where each token may refer to an operator, a keyword, a function etc. [3].…”

“…Source code author identification can be seen as a text classification task given that samples of known authorship by a set of candidate authors are available [2]. Burrows [3] presents an excellent review of software forensics applications associated with this task. These include assisting the revealing of academic dishonesty cases, resolving disputes and litigation about source code samples, tracing the authors of malicious software, and assisting the maintenance of large software projects by assigning source code samples to contributors.…”

“…Some very promising results have been reported when dealing with short samples of code, multiple candidate authors and several programming languages, including C, C++, Java, and Lisp [2,3,7]. To evaluate the proposed models, the published studies use custom-built source code collections that include either balanced training sets (where the training samples are equally distributed over the candidate authors) or imbalanced (skewed) training sets (where some candidate authors are overrepresented or under-represented in the training samples).…”

“…Therefore, a more accurate definition of a skewed training set should also account for the lines of code (or KBs) of training samples per author. Existing studies avoid focusing on the class imbalance problem and its effect on the performance of source code author identification methods [3,4,7].…”