Neural Program Repair with Execution-based Backpropagation

Abstract

Neural machine translation (NMT) architectures have achieved promising results for automatic program repair. Yet, they have the limitation of generating low-quality patches (e.g., not compilable patches). This is because the existing works only optimize a purely syntactic loss function based on characters and tokens without incorporating program-specific information during neural net weight optimization. In this paper, we propose a novel program repair model called RewardRepair. The core novelty of RewardRepair is to improve NMT-based program repair with a loss function based on program compilation and test execution information, rewarding the network to produce patches that compile and that do not overfit. We conduct several experiments to evaluate RewardRepair showing that it is feasible and effective to use compilation and test execution results to optimize the underlying neural repair model. In total, RewardRepair correctly repairs 43 Defects4J bugs including eight that are fixed for the first time.

CIRCLE: Continual Repair across Programming Languages

“…DL-based APRs have achieved state-of-the-art performance on program repair task [9,11,45,76,88]. Most of them treat repairing as a neural machine translation task and optimize an encoderdecoder model on a set of bug-fix pairs to learn latent patterns based on supervised learning.…”

A Retrospective on ICSE 2022

“…This is likely one of the reasons behind the low-quality patches generated through deep learning approaches. RewardRepair developed a new loss function that executes predicted patches and penalizes patches that do not compile and thereby enables improvement over state-of-the-art approaches [38]. These results are promising; however, a discussion with the authors of the above papers revealed that there are a number of uncertainties in the future of deep learning-based APR methods.…”

Self-Supervised Learning to Prove Equivalence Between Programs via Semantics-Preserving Rewrite Rules

“…For incremental learning in S4Eq, we use an "instance incremental scenario" in that for our problem we keep the output vocabulary constant while creating new data for incremental learning model updates [58]. Ye et al discuss using an output verification process (in their case compilation and test for program repair) to adjust the loss function in later training iterations [59]; our approach is related in that we test outputs but instead of adjusting the training loss we create new training samples which helps to generalize the model to a different problem domain.…”