Toward Imagined Speech based Smart Communication System: Potential Applications on Metaverse Conditions

Lee, Seo-Hyun; Lee, Young‐Eun; Lee, Seong–Whan

doi:10.1109/bci53720.2022.9734827

Cited by 21 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In [19], the authors give a comprehensive survey on computational arts that blend virtual and physical environment in the metaverse. The authors in [20] design a brain-to-speech scheme for smart communication in the real world, which is also presented as a potential application in the metaverse. The authors in [21] propose an operating system for the metaverse based on extended reality, which integrates hardware, computer vision and extended reality specific network.…”

Section: B Metaverse Servicesmentioning

confidence: 99%

Reliable Distributed Computing for Metaverse: A Hierarchical Game-Theoretic Approach

Jiang

Kang

Niyato

et al. 2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

The metaverse is regarded as a new wave of technological transformation that provides a virtual space for people to interact through digital avatars. To achieve immersive user experiences in the metaverse, real-time rendering is the key technology. However, computing intensive tasks of real-time rendering from metaverse service providers cannot be processed efficiently on a single resource-limited mobile device. Alternatively, such mobile devices can offload the metaverse rendering tasks to other mobile devices by adopting the collaborative computing paradigm based on Coded Distributed Computing (CDC). Therefore, this paper introduces a hierarchical game-theoretic CDC framework for the metaverse services, especially for vehicular metaverse. In the framework, idle resources from vehicles, acting as CDC workers, are aggregated to handle intensive computation tasks in the vehicular metaverse. Specifically, in the upper layer, a miner coalition formation game is formulated based on a reputation metric to select reliable workers. To guarantee the reliable management of reputation values, the reputation values calculated based on the subjective logical model are maintained in a blockchain database. In the lower layer, a Stackelberg game based incentive mechanism is considered to attract reliable workers selected in the upper layer to participate in render-

show abstract

Section: B Metaverse Servicesmentioning

confidence: 99%

Reliable Distributed Computing for Metaverse: A Hierarchical Game-Theoretic Approach

Jiang

Kang

Niyato

et al. 2023

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

show abstract

“…Applications of BCI to the Metaverse are still in its infancy. In [15], the authors proposed a framework for imagined speech communication in home control applications using BCI. The user's EEG signals were extracted and processed using a locally deployed SVM classifier to detect the user's commands.…”

Section: B Related Workmentioning

confidence: 99%

“…The results also showed that participants did not experience motion sickness, as the negativities of the locomotion movements were eliminated by using EEG signals instead of a joystick controller. It is worth noting that both studies in [12], [15] were pseudo-online and supported by wired connection between a computer and BCI headsets. The local computer was responsible for both processing and providing VR experiences.…”

Section: B Related Workmentioning

confidence: 99%

“…The mobility, scalability, and computation limitations of [12] and [15] could be addressed by utilizing wireless BCI headsets [17] connecting to a centralized server or an edge computing unit with much more abundant computational power. By this way, the remote server and the users can exchange information with each other through uplink and downlink wireless channels.…”

Section: B Related Workmentioning

confidence: 99%

“…By utilizing brain signals from users, we can potentially develop a human-centric Metaverse that authentically and individually represents humans through their brain activities, while significantly reducing the number of external sensors attached to the human body. Moreover, carrying brain signals over communication channels such as wireless mediums can open up new areas of application, such as brain communications, imagined speech, semantic communications, enabling users to communicate with one another through their thoughts e.g., [15]. To this end, Brain-Computer Interface (BCI) technology [16] provides a neural pathway between the human brain and the outside world through sensors attached to the human scalp, enabling the recording of brain activities and connection to computing units such as remote servers or personal computers.…”

Section: Introduction a Motivationmentioning

confidence: 99%

See 2 more Smart Citations

MetaChain: A Novel Blockchain-based Framework for Metaverse Applications

Nguyen

Hoang

Nguyen

et al. 2022

2022 IEEE 95th Vehicular Technology Conference: (VTC2022-Spring)

View full text Add to dashboard Cite

Decoding brain signals can not only reveal Metaverse users' expectations but also early detect error-related behaviors such as stress, drowsiness, and motion sickness. For that, this article proposes a pioneering framework using wireless/overthe-air Brain-Computer Interface (BCI) to assist creation of virtual avatars as human representation in the Metaverse. Specifically, to eliminate the computational burden for Metaverse users' devices, we leverage Wireless Edge Servers (WES) that are popular in 5G architecture and therein URLLC, enhanced broadband features to obtain and process the brain activities, i.e., electroencephalography (EEG) signals (via uplink wireless channels). As a result, the WES can learn human behaviors, adapt system configurations, and allocate radio resources to create individualized settings and enhance user experiences. Despite the potential of BCI, the inherent noisy/fading wireless channels and the uncertainty in Metaverse users' demands and behaviors make the related resource allocation and learning/classification problems particularly challenging. We formulate the joint learning and resource allocation problem as a Quality-of-Experience (QoE) maximization problem that takes into the latency, brain classification accuracy, and resources of the system. To tackle this mixed integer programming problem, we then propose two novel algorithms that are (i) a hybrid learning algorithm to maximize the user QoE and (ii) a meta-learning algorithm to exploit the neurodiversity of the brain signals among multiple Metaverse users. The extensive experiment results with different BCI datasets show that our proposed algorithms can not only provide low delay for virtual reality (VR) applications but also can achieve high classification accuracy for the collected brain signals.

show abstract