Typilus: neural type hints

Allamanis, Miltiadis; Barr, Earl T.; Ducousso, Soline; Gao, Zheng

doi:10.1145/3385412.3385997

Cited by 100 publications

(77 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Machine Learning for Software Engineering has seen rapid development in recent years inspired by the successful application in the Natural Language Processing field [17]. It is applied in many tasks related to software code such as code translation (e.g., [18]), type inference (e.g., [9,10]), code refactoring (e.g., [19]) and, as we list below, bug identification.…”

Section: Related Workmentioning

confidence: 99%

“…In future work, we also plan to make use of techniques such as Byte-Pair Encoding (BPE) [34,35], which attempts to mitigate the impact of out-of-vocabulary tokens. We note that the use of BPE is becoming more and more common in software engineering models (e.g., [10,33,36]).…”

Section: Future Workmentioning

confidence: 99%

“…However, recent advances in machine and deep learning have shown that models can learn useful information from the syntactic and semantic information that exist in source code. Tasks that were deemed not possible before, such as method naming [6,7,8], type inference [9,10,11], and bug finding [12], are now feasible. The lack of reliable tools that detect off-by-one mistakes leaves an excellent opportunity for researchers to experiment with machine learning approaches.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning Off-By-One Mistakes: An Empirical Study

Sellik

Paridon²,

Gousios

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

Mistakes in binary conditions are a source of error in many software systems. They happen when developers use, e.g., '<' or '>' instead of '<=' or '>='. These boundary mistakes are hard to find and impose manual, labor-intensive work for software developers.While previous research has been proposing solutions to identify errors in boundary conditions, the problem remains open. In this paper, we explore the effectiveness of deep learning models in learning and predicting mistakes in boundary conditions. We train different models on approximately 1.6M examples with faults in different boundary conditions. We achieve a precision of 85% and a recall of 84% on a balanced dataset, but lower numbers in an imbalanced dataset. We also perform tests on 41 real-world boundary condition bugs found from GitHub, where the model shows only a modest performance. Finally, we test the model on a large-scale Java code base from Adyen, our industrial partner. The model reported 36 buggy methods, but none of them were confirmed by developers.Index Terms-machine learning for software engineering, deep learning for software engineering, software testing, boundary testing.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Off-By-One Mistakes: An Empirical Study

Sellik

Paridon²,

Gousios

et al. 2021

2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)

View full text Add to dashboard Cite

show abstract

“…Type Inference: Debin also attempted to recover type information -which is a different problem. Typilus (Allamanis et al, 2020) is a new GGNN-based approach for type inference.…”

Section: Related Workmentioning

confidence: 99%

DIRECT : A Transformer-based Model for Decompiled Identifier Renaming

Saieva¹,

Ray²,

Kaiser³

2021

Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021)

View full text Add to dashboard Cite

Decompiling binary executables to high-level code is an important step in reverse engineering scenarios, such as malware analysis and legacy code maintenance. However, the generated high-level code is difficult to understand since the original variable names are lost. In this paper, we leverage transformer models to reconstruct the original variable names from decompiled code. Inherent differences between code and natural language present certain challenges in applying conventional transformer-based architectures to variable name recovery. We propose DIRECT, a novel transformer-based architecture customized specifically for the task at hand. We evaluate our model on a dataset of decompiled functions and find that DIRECT outperforms the previous state-of-the-art model by up to 20%. We also present ablation studies evaluating the impact of each of our modifications. We make the source code of DIRECT available to encourage reproducible research.

show abstract

“…We then compared the types inferred by SimTyper to existing, programmer-written annotations. Following prior work [Allamanis et al 2020], we count the number of cases where the the inferred type matches the existing type, as well as the number of matches up to parameterÐgeneric types where the base matches but the parameter is different (e.g., inferring Array<Integer> when the original was Array<String> would fall in this category). We found that, by combining constraint solving, InferDL heuristics, and type equality prediction from DeepSim, SimTyper generated 66% more type annotations that matched programmer-written types compared to using constraint solving alone.…”

Section: Introductionmentioning

confidence: 99%

SimTyper: sound type inference for Ruby using type equality prediction

Kazerounian

Foster

Min

2021

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Many researchers have explored type inference for dynamic languages. However, traditional type inference computes most general types which, for complex type systems—which are often needed to type dynamic languages—can be verbose, complex, and difficult to understand. In this paper, we introduce SimTyper, a Ruby type inference system that aims to infer usable types—specifically, nominal and generic types—that match the types programmers write. SimTyper builds on InferDL, a recent Ruby type inference system that soundly combines standard type inference with heuristics. The key novelty of SimTyper is type equality prediction , a new, machine learning-based technique that predicts when method arguments or returns are likely to have the same type. SimTyper finds pairs of positions that are predicted to have the same type yet one has a verbose, overly general solution and the other has a usable solution. It then guesses the two types are equal, keeping the guess if it is consistent with the rest of the program, and discarding it if not. In this way, types inferred by SimTyper are guaranteed to be sound. To perform type equality prediction, we introduce the deep similarity (DeepSim) neural network. DeepSim is a novel machine learning classifier that follows the Siamese network architecture and uses CodeBERT, a pre-trained model, to embed source tokens into vectors that capture tokens and their contexts. DeepSim is trained on 100,000 pairs labeled with type similarity information extracted from 371 Ruby programs with manually documented, but not checked, types. We evaluated SimTyper on eight Ruby programs and found that, compared to standard type inference, SimTyper finds 69% more types that match programmer-written type information. Moreover, DeepSim can predict rare types that appear neither in the Ruby standard library nor in the training data. Our results show that type equality prediction can help type inference systems effectively produce more usable types.

show abstract

Typilus: neural type hints

Cited by 100 publications

References 38 publications

Learning Off-By-One Mistakes: An Empirical Study

Learning Off-By-One Mistakes: An Empirical Study

DIRECT : A Transformer-based Model for Decompiled Identifier Renaming

SimTyper: sound type inference for Ruby using type equality prediction

Contact Info

Product

Resources

About