EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs)

Acqualagna, Laura; Bosse, Sebastian; Porbadnigk, Anne K.; Curio, Gabriel; Müller, Klaus Robert; Wiegand, Thomas; Blankertz, Benjamin

doi:10.1088/1741-2560/12/2/026012

Cited by 51 publications

(37 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The potential for a direct measurement [30] has successfully been exploited in traditional settings; see, e.g., [29,31,32]. Typically, EEG measurements at 300-500 ms after the stimulus, such as media display or quality changes, are utilized, with potential drawbacks [33]. In typical Brain-Computer Interface (BCI) research approaches, larger numbers of wet electrodes are utilized in human subject experimentation within clinical settings.…”

Section: Related Workmentioning

confidence: 99%

Visual Interface Evaluation for Wearables Datasets: Predicting the Subjective Augmented Vision Image QoE and QoS

Bauman

Seeling

2017

Future Internet

View full text Add to dashboard Cite

Abstract:As Augmented Reality (AR) applications become commonplace, the determination of a device operator's subjective Quality of Experience (QoE) in addition to objective Quality of Service (QoS) metrics gains importance. Human subject experimentation is common for QoE relationship determinations due to the subjective nature of the QoE. In AR scenarios, the overlay of displayed content with the real world adds to the complexity. We employ Electroencephalography (EEG) measurements as the solution to the inherent subjectivity and situationality of AR content display overlaid with the real world. Specifically, we evaluate prediction performance for traditional image display (AR) and spherical/immersive image display (SAR) for the QoE and underlying QoS levels. Our approach utilizing a four-position EEG wearable achieves high levels of accuracy. Our detailed evaluation of the available data indicates that less sensors would perform almost as well and could be integrated into future wearable devices. Additionally, we make our Visual Interface Evaluation for Wearables (VIEW) datasets from human subject experimentation publicly available and describe their utilization.

show abstract

Section: Related Workmentioning

confidence: 99%

Visual Interface Evaluation for Wearables Datasets: Predicting the Subjective Augmented Vision Image QoE and QoS

Bauman

Seeling

2017

Future Internet

View full text Add to dashboard Cite

show abstract

“…Additionally, all of these studies have in common that they use very short video stimuli for their experimental paradigms, as well as lots of repetitions (which is mostly due to the nature of an ERP) and thus do not conform with standard quality recommendations [6] [42] nor with realistic settings. In the study of Acqualagna [20], the SSVEP, a different feature of the EEG signal, was used and could confirm prior results. As for SSVEPs, a lower number of repetitions necessary for this paradigm might be better suited for quality assessment of still images.…”

Section: Authormentioning

confidence: 84%

“…In Acqualagna et al [20], a set of six different images with gray level texture was selected. Each image was presented at seven different compression levels produced by the HM10.0 HVEC codec (including one reference condition).…”

Section: Authormentioning

confidence: 99%

Review on using physiology in quality of experience

Arndt¹,

Brunnström²,

Cheng³

et al. 2016

View full text Add to dashboard Cite

In the area of Quality of Experience (QoE), one challenge is to design test methodologies in order to evaluate the perceived quality of multimedia content delivered through technical systems. Traditionally, this evaluation is done using subjective opinion tests. However, sometimes it is difficult for observers to communicate the experienced quality through the given scale. Furthermore, those tests do not give insights into how the user is reacting on an internal physiological level. To overcome these issues, one approach is to use physiological measures, in order to derive a direct non-verbal response of the recipient. In this paper, we review studies that have been performed in the domain of QoE using physiological measures and we look into current activities in standardization bodies. We present challenges this research faces, and give an overview on what researchers should be aware of when they want to start working in this research area.

show abstract

“…In contexts where the trial-to-trial variability of EEG is not viewed as a notorious obstacle for analysis, but as a source of information, LRP can contribute high-resolving spatio-temporal representations of underlying neurophysiological phenomena. In particular, this might be interesting for linking brain indices to single instances of behavioral measures [19], for understanding subtle aspects of complex perceptual processes, such as perception of video or audio quality [20], [21], and of dynamic cognitive processes, such as decision making [22]. Finally, a trained network produces relevance maps for any (even artificially generated) DNN decision.…”

Section: Discussionmentioning

confidence: 99%

Interpretable deep neural networks for single-trial EEG classification

Sturm

Lapuschkin

Samek

et al. 2016

Journal of Neuroscience Methods

Self Cite

339

235

View full text Add to dashboard Cite

Abstract-Background: In cognitive neuroscience the potential of Deep Neural Networks (DNNs) for solving complex classification tasks is yet to be fully exploited. The most limiting factor is that DNNs as notorious 'black boxes' do not provide insight into neurophysiological phenomena underlying a decision. Layerwise Relevance Propagation (LRP) has been introduced as a novel method to explain individual network decisions. New Method: We propose the application of DNNs with LRP for the first time for EEG data analysis. Through LRP the singletrial DNN decisions are transformed into heatmaps indicating each data point's relevance for the outcome of the decision. Results: DNN achieves classification accuracies comparable to those of CSP-LDA. In subjects with low performance subjectto-subject transfer of trained DNNs can improve the results. The single-trial LRP heatmaps reveal neurophysiologically plausible patterns, resembling CSP-derived scalp maps. Critically, while CSP patterns represent class-wise aggregated information, LRP heatmaps pinpoint neural patterns to single time points in single trials. Comparison with Existing Method(s):We compare the classification performance of DNNs to that of linear CSP-LDA on two data sets related to motor-imaginery BCI. Conclusion: We have demonstrated that DNN is a powerful nonlinear tool for EEG analysis. With LRP a new quality of highresolution assessment of neural activity can be reached. LRP is a potential remedy for the lack of interpretability of DNNs that has limited their utility in neuroscientific applications. The extreme specificity of the LRP-derived heatmaps opens up new avenues for investigating neural activity underlying complex perception or decision-related processes.

show abstract

EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs)

Abstract: The results show that neural assessment of video quality based on SSVEPs is a viable complement of the behavioral one and a significantly fast alternative to methods based on the P3 component.

Cited by 51 publications

References 75 publications

Visual Interface Evaluation for Wearables Datasets: Predicting the Subjective Augmented Vision Image QoE and QoS

Visual Interface Evaluation for Wearables Datasets: Predicting the Subjective Augmented Vision Image QoE and QoS

Review on using physiology in quality of experience

Interpretable deep neural networks for single-trial EEG classification

Contact Info

Product

Resources

About