Is Knowledge the Key? An Experiment on Debiasing Architectural Decision-Making - a Pilot Study

Debiasing architectural decision-making: a workshop-based training approach

“…The experiment plan has been made available online [2]. While planning the experiment, we enhanced most steps from our previous approach [1] to both improve the debiasing treatment itself and the validity of the experiment. The basic steps of the experiment included:…”

“…These specific techniques were proposed previously as a result of our previous work [1] where we analysed, in detail, how each of the three researched biases usually impacted the teams that took part in the study. However, during that study, the effectiveness of these techniques was not validated.…”

“…Analysis For the analysis, we used a modified approach of our method from our previous study [1]. In order to analyse the results, we transcribed all recordings of the tasks, during which the participants' created their design.…”

“…Notably, Shepperd et al [12] proposed a successful treatment that improved software effort estimation, through a two-to threehour-long workshop about cognitive biases. Our team attempted an empirical validation of an anti-bias treatment for ADM [1], but it turned out not to be successful. However, it had several major weaknesses:…”

“…Architectural decision-making in particular is not exempt from the influence of cognitive biases [14], [18]. However, research on debiasing architectural decision-making is scarce [9], with a clear lack of empirically proven debiasing methods that could be used in practice [10] In this paper, we endeavour to create an effective debiasing treatment, through expanding on our previous work [1]. The debiasing treatment that we designed consists of an hour-long workshop during which individuals learn about cognitive biases in architectural decision-making (ADM) and take part in three practical exercises.…”