Would You Fix This Code for Me? Effects of Repair Source and Commenting on Trust in Code Repair

Abstract

Automation and autonomous systems are quickly becoming a more engrained aspect of modern society. The need for effective, secure computer code in a timely manner has led to the creation of automated code repair techniques to resolve issues quickly. However, the research to date has largely ignored the human factors aspects of automated code repair. The current study explored trust perceptions, reuse intentions, and trust intentions in code repair with human generated patches versus automated code repair patches. In addition, comments in the headers were manipulated to determine the effect of the presence or absence of comments in the header of the code. Participants were 51 programmers with at least 3 years’ experience and knowledge of the C programming language. Results indicated only repair source (human vs. automated code repair) had a significant influence on trust perceptions and trust intentions. Specifically, participants consistently reported higher levels of perceived trustworthiness, intentions to reuse, and trust intentions for human referents compared to automated code repair. No significant effects were found for comments in the headers.

A Comprehensive Study of Code-removal Patches in Automated Program Repair

“…Tyler et al [34] show that developers in general tend to trust more in patches produced by humans than automatic program repair techniques. Similar results are the ones obtained by Alarcon et al [1], indeed, they report that programmers find human repairs more trustworthy than the ones generated by GenProg. The study by Cambronero et al [3] investigates how humans are used to use the patches automatically generated by program repair techniques.…”

A comprehensive study of code-removal patches in automated program repair

“…Ryan et al (2019) show that developers in general tend to trust more in patches produced by humans than automatic program repair techniques. Similar results are the ones obtained by Alarcon et al (2020), indeed, they report that programmers find human repairs more trustworthy than the ones generated by GenProg. The study by Cambronero et al (2019) investigates how humans are used to use the patches automatically generated by program repair techniques.…”

Let’s Talk With Developers, Not About Developers: A Review of Automatic Program Repair Research

“…The survey results also provide indications of what might increase developer trust in automatically generated patches, such as test cases, explanations of the patch, and evidence of patch correctness. Alarcon et al's experimental study also considered trust in APR, and found that the source of the repair (human vs. automated) had significant influence on trust perceptions and intentions, participants having higher trust in human repairs than automated repairs [22]. Both these studies demonstrate recent advances in human factors research in APR.…”