Building predictive models of emotion with functional near-infrared spectroscopy

Bandara, Danushka; Velipasalar, Senem; Bratt, Sarah; Hirshfield, Leanne M.

doi:10.1016/j.ijhcs.2017.10.001

Cited by 40 publications

(32 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…processing during aesthetic experiences [31]. Functional near-infrared spectroscopy (fNIRS) demonstrated the capability of discriminating between affective states on the valence and arousal dimensions [32]. fNIRS uses near-infrared light with spectroscopy principles.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Neuroticism is associated with reduced oxygenation levels in the lateral prefrontal cortex following exposure to unpleasant images

Balada

Lucas

Blanch

et al. 2019

Physiology & Behavior

View full text Add to dashboard Cite

show abstract

Section: Accepted Manuscriptmentioning

confidence: 99%

Neuroticism is associated with reduced oxygenation levels in the lateral prefrontal cortex following exposure to unpleasant images

Balada

Lucas

Blanch

et al. 2019

Physiology & Behavior

View full text Add to dashboard Cite

show abstract

“…This is mainly due to the inability of the traditional machine learning algorithms like Naı¨ve Bayes, to incorporate spatial data without some specialized preprocessing. 40 Furthermore, the majority of fNIR statistical and machine learning techniques are applied to just the HbO data, as HbO is known to have the strongest response to neural activation.…”

Section: Functional Near Infrared Spectroscopymentioning

confidence: 99%

“…Since the process of collecting training data with brain measurement devices is costly and time-consuming, researchers have noted the need to build models across participants, where an adequate amount of data can be used for model training, 38,39 and it is becoming more commonplace to build and evaluate models using ‘leave-one-participant-out’ or ‘leave-one-fold-out’ cross-validation. 39–41 However, individual brain differences regarding the spatial and temporal complexities of the human brain remain a formidable challenge for this research area.…”

Section: Background and Related Workmentioning

confidence: 99%

Classification of affect using deep learning on brain blood flow data

Bandara

Hirshfield

Velipasalar

2019

Journal of Near Infrared Spectroscopy

Self Cite

View full text Add to dashboard Cite

We present a convolutional neural network-and long short-term memory-based method to classify the valence level of a computer user based on functional near infrared spectroscopy data. Convolutional neural networks are well suited for capturing the spatial characteristics of functional near infrared spectroscopy data. And long short-term memories are demonstrated to be good at learning temporal patterns of unknown length in time series data. We explore these methods in a combined layered architecture in order to improve classification accuracy. We conducted an experiment with 20 participants, wherein they were subjected to emotion inducing stimuli while their brain activity was measured using functional near infrared spectroscopy. Self-report surveys were administered after each stimulus to gauge participants' self-assessment of their valence. The resulting classification using these survey labels as ground truth provided a three-class classification accuracy 77.89% in across subject cross-validation. This method also shows promise for generalization to other classification tasks using functional near infrared spectroscopy data.

show abstract

“…However, for neuroergonomic studies in specific, which focus on understanding the human brain functions in everyday life settings, the confinement of a scanner environment is not ideal. In this regard, fNIRS, another optical neuroimaging modality, is fast emerging as a de-facto choice for imaging the human brain for neuroergonomic studies owing to its non-invasive, highly portable and wearable characteristics [ 19 , 20 ]. fNIRS uses near infra-red (NIR) light to read cerebral activity by introducing NIR at the specific location of interest over the scalp.…”

Section: Introductionmentioning

confidence: 99%

Single-Trial Recognition of Video Gamer’s Expertise from Brain Haemodynamic and Facial Emotion Responses

et al. 2021

View full text Add to dashboard Cite

With an increase in consumer demand of video gaming entertainment, the game industry is exploring novel ways of game interaction such as providing direct interfaces between the game and the gamers’ cognitive or affective responses. In this work, gamer’s brain activity has been imaged using functional near infrared spectroscopy (fNIRS) whilst they watch video of a video game (League of Legends) they play. A video of the face of the participants is also recorded for each of a total of 15 trials where a trial is defined as watching a gameplay video. From the data collected, i.e., gamer’s fNIRS data in combination with emotional state estimation from gamer’s facial expressions, the expertise level of the gamers has been decoded per trial in a multi-modal framework comprising of unsupervised deep feature learning and classification by state-of-the-art models. The best tri-class classification accuracy is obtained using a cascade of random convolutional kernel transform (ROCKET) feature extraction method and deep classifier at 91.44%. This is the first work that aims at decoding expertise level of gamers using non-restrictive and portable technologies for brain imaging, and emotional state recognition derived from gamers’ facial expressions. This work has profound implications for novel designs of future human interactions with video games and brain-controlled games.

show abstract

Building predictive models of emotion with functional near-infrared spectroscopy

Cited by 40 publications

References 50 publications

Neuroticism is associated with reduced oxygenation levels in the lateral prefrontal cortex following exposure to unpleasant images

Neuroticism is associated with reduced oxygenation levels in the lateral prefrontal cortex following exposure to unpleasant images

Classification of affect using deep learning on brain blood flow data

Single-Trial Recognition of Video Gamer’s Expertise from Brain Haemodynamic and Facial Emotion Responses

Contact Info

Product

Resources

About