On the Placebo Effect in Interactive SBSE: A Preliminary Study

Souza, Jerffeson; Araújo, Allysson Allex; Yeltsin, Italo; Saraiva, Raphael; Soares, Pamella

doi:10.1007/978-3-319-99241-9_23

Cited by 1 publication

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another study [23] investigates how the subjective evaluation of an automatically generated solution is positively influenced by involving the decision maker in the process but not considering their decisions at all. They conducted a placebocontrolled study with 12 software engineering practitioners and found an increase of 68% in the subjective evaluation of an automatically generated but supposedly decision influenced solution is due to a placebo effect.…”

Section: A Placebo Effectmentioning

confidence: 99%

The Mind Is a Powerful Place: How Showing Code Comprehensibility Metrics Influences Code Understanding

Wyrich

Preikschat

Graziotin

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Static code analysis tools and integrated development environments present developers with quality-related software metrics, some of which are on how understandable their source code is. Software metrics influence overarching strategic decisions that impact the future of companies and the prioritization of everyday software development tasks. Several software metrics, however, lack in validation: we just choose to trust that they reflect what they are supposed to measure. Some of them were even shown to not measure the quality aspects they intend to measure. Yet, they influence us through biases in our cognitive-driven actions. In particular, they might anchor us in our decisions. Whether the anchoring effect exists with software metrics has not been studied yet. We conduct a randomized and double-blind experiment to investigate the extent to which a displayed metric value for source code comprehensibility anchors developers in their subjective rating of source code comprehensibility, whether performance is affected by the anchoring effect when working on comprehension tasks, and which individual characteristics might play a role in the anchoring effect. We found that the displayed value of a comprehensibility metric has a significant and large anchoring effect on a developer's code comprehensibility rating. The effect does not seem to affect the time or correctness when working on comprehension questions related to the code snippet under study. Since the anchoring effect is one of the most robust cognitive biases, and we have limited understanding of the consequences of the demonstrated manipulation of developers by non-validated metrics, we call for an increased awareness of the responsibility in code quality reporting and for corresponding tools to be based on scientific evidence. 1

show abstract

Section: A Placebo Effectmentioning

confidence: 99%