Spotting Problematic Code Lines using Nonintrusive Programmers' Biofeedback

Couceiro, Ricardo; Carvalho, Paulo; Madeira, Henrique; Barbosa, Raul; Duraes, Joao; Duarte, Goncalo; Castelhano, João; Duarte, Catarina; Teixeira, César; Laranjeiro, Nuno; Medeiros, Julio

doi:10.1109/issre.2019.00019

Cited by 16 publications

(16 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent results appeared in [6] in the context of assessing cognitive state (mainly high cognitive load and distraction states) of software programmers show that it is possible to associate such cognitive information, in real-time, to specific lines of code or lexical tokens, to identify such code lines as being more susceptible of having software bugs. In the work reported herein, we will expand such preliminary results obtained in the specific context of prevention of software bugs, to a much broader context of general content comprehension.…”

Section: Methodsologymentioning

confidence: 99%

“…Eye-tracking devices have also been used in research to study the eye gaze during distinct mental tasks such as code comprehension [4,5,6]. In principle, eye-tracking is the process of tracking the eye movement and determining where the user is looking at or the absolute point of gaze (POG), which refers to the point in the visual scene at which the user's gaze is focused on.…”

Section: B Pupillography and Eye-trackingmentioning

confidence: 99%

“…The dataset that was used for this study is an already existing publicly available dataset, which was originated from the BASE project mentioned in [4,5,6]. The dataset was developed in one of the project's studies to monitor physiologic reactions and mental effort that are associated with code comprehension in different complexities using non-intrusive biosensors.…”

Section: A Datasetmentioning

confidence: 99%

“…Biofeedback devices have been extensively used in research to study the cognitive process involved in various activities such as software development (e.g., see a recent survey on measuring the cognitive load of software developers [9]). Recent work in the context of programmer's errors in software development [4,5,6] showed that the integration of HRV, pupillometry, and eyetracking allows the identification of the code lines that correspond to an increase of cognitive load of individual programmers. Although the goal of the research published in [4,5,6] is the annotation of source code for the prediction of programmers' errors, it shows that it is possible to associate programmers' mental load in real-time to specific lines of code or lexical tokens [6].…”

Section: Introductionmentioning

confidence: 99%

“…Recent work in the context of programmer's errors in software development [4,5,6] showed that the integration of HRV, pupillometry, and eyetracking allows the identification of the code lines that correspond to an increase of cognitive load of individual programmers. Although the goal of the research published in [4,5,6] is the annotation of source code for the prediction of programmers' errors, it shows that it is possible to associate programmers' mental load in real-time to specific lines of code or lexical tokens [6]. This has motivated us to explore the same concept in the much broader context of general content comprehension and intelligent user interfaces, as proposed in TellBack.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Intelligent Biofeedback Augmented Content Comprehension (TellBack)

et al. 2021

Self Cite

View full text Add to dashboard Cite

Assessing comprehension difficulties requires the ability to assess cognitive load. Changes in cognitive load induced by comprehension difficulties could be detected with an adequate time resolution using different biofeedback measures (e.g., changes in the pupil diameter). However, identifying the Spatiotemporal sources of content comprehension difficulties (i.e., when, and where exactly the difficulty occurs in content regions) with a fine granularity is a big challenge that has not been explicitly addressed in the state-of-the-art. This paper proposes and evaluates an innovative approach named Intelligent Biofeedback Augmented Content Comprehension (TellBack) to explicitly address this challenge. The goal is to autonomously identify regions of digital content that cause user's comprehension difficulty, opening the possibility to provide real-time comprehension support to users. TellBack is based on assessing the cognitive load associated with content comprehension through non-intrusive cheap biofeedback devices that acquire measures such as pupil response or Heart Rate Variability (HRV). To identify when exactly the difficulty in comprehension occurs, physiological manifestations of the Autonomic Nervous System (ANS) such as the pupil diameter variability and the modulation of HRV are exploited, whereas the fine spatial resolution (i.e., the region of content where the user is looking at) is provided by eye-tracking. The evaluation results of this approach show an accuracy of 83.00% ± 0.75 in classifying regions of content as difficult or not difficult using Support Vector Machine (SVM), and precision, recall, and micro F1-score of 0.89, 0.79, and 0.83, respectively. Results obtained with 4 other classifiers, namely Random Forest, knearest neighbor, Decision Tree, and Gaussian Naïve Bayes, showed a slightly lower precision. TellBack outperforms the state-of-the-art in precision & recall by 23% and 17% respectively.

show abstract

Section: Methodsologymentioning

confidence: 99%

Section: B Pupillography and Eye-trackingmentioning

confidence: 99%

Section: A Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Intelligent Biofeedback Augmented Content Comprehension (TellBack)

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Neurophysiological Measurements in the Research Field of Interruption Science: Insights into Applied Methods for Different Interruption Types Based on an Umbrella Review

Stangl,

Riedl

2024

Information Systems and Neuroscience

View full text Add to dashboard Cite

A family of experiments about how developers perceive delayed system response time

Cornejo,

Briola,

Micucci

et al. 2024

Software Qual J

View full text Add to dashboard Cite

Collecting and analyzing data about developers working on their development tasks can help improve development practices, finally increasing the productivity of teams. Indeed, monitoring and analysis tools have already been used to collect data from productivity tools. Monitoring inevitably consumes resources and, depending on their extensiveness, may significantly slow down software systems, interfering with developers’ activity. There is thus a challenging trade-off between monitoring and validating applications in their operational environment and preventing the degradation of the user experience. The lack of studies about when developers perceive an overhead introduced in an application makes it extremely difficult to fine-tune techniques working in the field. In this paper, we address this challenge by presenting an empirical study that quantifies how developers perceive overhead. The study consists of three replications of an experiment that involved 99 computer science students in total, followed by a small-scale experimental assessment of the key findings with 12 professional developers. Results show that non-negligible overhead can be introduced for a short period into applications without developers perceiving it and that the sequence in which complex operations are executed influences the perception of the system response time. This information can be exploited to design better monitoring techniques.

show abstract

Spotting Problematic Code Lines using Nonintrusive Programmers' Biofeedback

Cited by 16 publications

References 27 publications

Intelligent Biofeedback Augmented Content Comprehension (TellBack)

Intelligent Biofeedback Augmented Content Comprehension (TellBack)

Neurophysiological Measurements in the Research Field of Interruption Science: Insights into Applied Methods for Different Interruption Types Based on an Umbrella Review

A family of experiments about how developers perceive delayed system response time

Contact Info

Product

Resources

About