An Open Source Multi-Modal Data-Acquisition Platform for Experimental Investigation of Blended Control of Scale Vehicles

Abstract

Currently many autonomous vehicles require a person to monitor the system and take over when an unexpected or unusual event occurs. A person may be able to monitor multiple such vehicles but a question arises as to how many, and how to measure the cognitive requirements. Brain Computer Interfaces (BCI) operating passively could aid in assisting remote operators in such tasks but there is as yet significant research to be undertaken before such technology becomes robust and effective. To this end we describe a platform for acquisition of multi-modal data for passive hybrid Brain Computer Interface (phBCI) development purposes. The open source system integrates electroencephalography (EEG), computer vision and a wearable inertial measurement unit (IMU) along with timestamped event markers for a subject engaged in a set of driving-related tasks as applied to blended control of multiple vehicles. The vehicular control task is realised both with graded complexity simulations and physical scale autonomous vehicles. This platform has the following significant advantages: reduced experimental variability due to data acquisition system implementation decisions; ease of reproduction of experiments through shareable configuration information; and acceleration of open science dataset accumulation. Consequently this freely available open source platform has the potential to enhance the reproducibility of passive hybrid BCI experimental research.

AMBER: advancing multimodal brain-computer interfaces for enhanced robustness—A dataset for naturalistic settings

“…By understanding the relationship between facial expressions, head movements, and brain activity, researchers can gain insights into emotion recognition, attention, and cognitive processes. Moreover, this approach can also be applied to develop advanced human-computer interfaces and improve the accuracy of brain-computer interfaces (Redmond et al, 2022 ).…”