Crowdsourcing Affective Annotations Via fNIRS-BCI

Ruotsalo, Tuukka; Mäkelä, Kalle; Spapé, Michiel M.

doi:10.1109/taffc.2023.3273916

Cited by 4 publications

(1 citation statement)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using fNIRS, brain activity is measured through hemodynamic responses associated with neuronal behavior ( Kamran and Hong, 2013 ; Santosa et al, 2013 ; Khan et al, 2014 ; Ruotsalo et al, 2023 ). The fNIRS can provide topographic ( Obrig and Villringer, 2003 ; Wolf et al, 2007 ; Hu et al, 2011 ; Li et al, 2018 ) and tomographic brain images ( Bluestone et al, 2001 ; Boas et al, 2004 ).…”

Section: Introductionmentioning

confidence: 99%

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Bhutta,

Ali,

Zafar

et al. 2024

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Deception is an inevitable occurrence in daily life. Various methods have been used to understand the mechanisms underlying brain deception. Moreover, numerous efforts have been undertaken to detect deception and truth-telling. Functional near-infrared spectroscopy (fNIRS) has great potential for neurological applications compared with other state-of-the-art methods. Therefore, an fNIRS-based spontaneous lie detection model was used in the present study. We interviewed 10 healthy subjects to identify deception using the fNIRS system. A card game frequently referred to as a bluff or cheat was introduced. This game was selected because its rules are ideal for testing our hypotheses. The optical probe of the fNIRS was placed on the subject’s forehead, and we acquired optical density signals, which were then converted into oxy-hemoglobin and deoxy-hemoglobin signals using the Modified Beer–Lambert law. The oxy-hemoglobin signal was preprocessed to eliminate noise. In this study, we proposed three artificial neural networks inspired by deep learning models, including AlexNet, ResNet, and GoogleNet, to classify deception and truth-telling. The proposed models achieved accuracies of 88.5%, 88.0%, and 90.0%, respectively. These proposed models were compared with other classification models, including k-nearest neighbor, linear support vector machines (SVM), quadratic SVM, cubic SVM, simple decision trees, and complex decision trees. These comparisons showed that the proposed models performed better than the other state-of-the-art methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Bhutta,

Ali,

Zafar

et al. 2024

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

Sneaky emotions: impact of data partitions in affective computing experiments with brain-computer interfacing

Moreno-Alcayde,

Traver,

Leiva

2023

Biomed. Eng. Lett.

View full text Add to dashboard Cite

Brain-Computer Interfacing (BCI) has shown promise in Machine Learning (ML) for emotion recognition. Unfortunately, how data are partitioned in training/test splits is often overlooked, which makes it difficult to attribute research findings to actual modeling improvements or to partitioning issues. We introduce the “data transfer rate” construct (i.e., how much data of the test samples are seen during training) and use it to examine data partitioning effects under several conditions. As a use case, we consider emotion recognition in videos using electroencephalogram (EEG) signals. Three data splits are considered, each representing a relevant BCI task: subject-independent (affective decoding), video-independent (affective annotation), and time-based (feature extraction). Model performance may change significantly (ranging e.g. from 50% to 90%) depending on how data is partitioned, in classification accuracy. This was evidenced in all experimental conditions tested. Our results show that (1) for affective decoding, it is hard to achieve performance above the baseline case (random classification) unless some data of the test subjects are considered in the training partition; (2) for affective annotation, having data from the same subject in training and test partitions, even though they correspond to different videos, also increases performance; and (3) later signal segments are generally more discriminative, but it is the number of segments (data points) what matters the most. Our findings not only have implications in how brain data are managed, but also in how experimental conditions and results are reported.

show abstract

Crowdsourcing with the drift diffusion model of decision making

Lalvani,

Katsaggelos

2024

Sci Rep

View full text Add to dashboard Cite

Crowdsourcing involves the use of annotated labels with unknown reliability to estimate ground truth labels in datasets. A common task in crowdsourcing involves estimating reliabilities of annotators (such as through the sensitivities and specificities of annotators in the binary label setting). In the literature, beta or dirichlet distributions are typically imposed as priors on annotator reliability. In this study, we investigated the use of a neuroscientifically validated model of decision making, known as the drift-diffusion model, as a prior on the annotator labeling process. Two experiments were conducted on synthetically generated data with non-linear (sinusoidal) decision boundaries. Variational inference was used to predict ground truth labels and annotator related parameters. Our method performed similarly to a state-of-the-art technique (SVGPCR) in prediction of crowdsourced data labels and prediction through a crowdsourced-generated Gaussian process classifier. By relying on a neuroscientifically validated model of decision making to model annotator behavior, our technique opens the avenue of predicting neuroscientific biomarkers of annotators, expanding the scope of what may be learnt about annotators in crowdsourcing tasks.

show abstract

Crowdsourcing Affective Annotations Via fNIRS-BCI

Cited by 4 publications

References 70 publications

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Artificial neural network models: implementation of functional near-infrared spectroscopy-based spontaneous lie detection in an interactive scenario

Sneaky emotions: impact of data partitions in affective computing experiments with brain-computer interfacing

Crowdsourcing with the drift diffusion model of decision making

Contact Info

Product

Resources

About