On the localness of software

Abstract

The n-gram language model, which has its roots in statistical natural language processing, has been shown to successfully capture the repetitive and predictable regularities ("naturalness") of source code, and help with tasks such as code suggestion, porting, and designing assistive coding devices. However, we show in this paper that this natural-language-based model fails to exploit a special property of source code: localness. We find that human-written programs are localized: they have useful local regularities that can be captured and exploited. We introduce a novel cache language model that consists of both an n-gram and an added "cache" component to exploit localness. We show empirically that the additional cache component greatly improves the n-gram approach by capturing the localness of software, as measured by both cross-entropy and suggestion accuracy. Our model's suggestion accuracy is actually comparable to a state-of-the-art, semantically augmented language model; but it is simpler and easier to implement. Our cache language model requires nothing beyond lexicalization, and thus is applicable to all programming languages.

On the "naturalness" of buggy code

“…Following this work, language models have been used to good effect in code suggestion [22,48,53,15], cross-language porting [38,37,39,24], coding standards [2], idiom mining [3], and code deobfuscation [47]. Since language models are useful in these tasks, ⇤ Baishakhi Ray and Vincent Hellendoorn are both first authors, and contributed equally to the work.…”

Stepwise API usage assistance using n -gram language models

“…Tu et al [28] however, argue that code tokenization is enough for the n-gram language models. They also argue that n-gram models will not be useful when a particular context is not present in the source code corpora used to train the model.…”

Multi-token code suggestions using statistical language models

“…Tu et al [3] sought to confirm that software is localized 2 Working on top of the fact that software is natural, they sought to find that there are "local regularities [in software] than be captured and exploited." They found, empirically, that this is the case.…”

“…Since Gamboge uses a simple n-gram model, extending its prediction backend with the cache language module developed by Tu et al [3] may be beneficial. Since they have already shown that in single-token contexts, it surpasses a bare n-gram language model in suggestion performance, it may also improve suggestion performance for multi-token predicting.…”

“…As mentioned in [1] and [3], naturalness and localness have several applications that are being lost on the crowd that are trying to make better code suggestion engines. However, in order to maintain large, and useful language models for the most popular languages, the storage and computation of such models may be used as part of an online service that, ostensibly, is intended only for providing robust code completion.…”