Comparison of SSVEP BCI and Eye Tracking for Controlling a Humanoid Robot in a Social Environment

Kishore, Sameer; González-Franco, Mar; Hintemüller, Christoph; Kapeller, Christoph; Guger, Christoph; Slater, Mel; Blom, Kristopher J.

doi:10.1162/pres_a_00192

Cited by 51 publications

(46 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition to the questions related to task performance, the visitors were also asked questions related to the BOI and agency twice, once for each robot ( Table 2). The embodiment questionnaire, designed based on a previous study that was carried out using the same robot (Kishore et al, 2014), was not given for VD3 since high ownership illusions and high levels of agency have been reported in various previous studies using the exact same setup (Banakou et al, 2013;Kilteni et al, 2013;Peck et al, 2013). The locals were asked two questions regarding their experience (Table 3).…”

Section: Methodsmentioning

confidence: 99%

“…One way of achieving agency is to have full-body tracking of all movements produced by the participant and mapping these to the corresponding limbs of an avatar or a robot in real time. This has been applied to generation of a body ownership illusion (BOI) over a humanoid robot body seen from first person perspective (Kishore et al, 2014), using the algorithm described in Spanlang et al (2013). Methods have also been developed to modify the appearance of a humanoid robot, by projecting a 3D model of the head of the participant on the face of the remote robot using the shader lamp method (Raskar et al, 2001).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Multi-Destination Beaming: Apparently Being in Three Places at Once through Robotic and Virtual Embodiment

et al. 2016

Self Cite

View full text Add to dashboard Cite

It has been shown that an illusion of ownership over an artificial limb or even an entire body can be induced in people through multisensory stimulation, providing evidence that the surrogate body is the person's actual body. Such body ownership illusions (BOIs) have been shown to occur with virtual bodies, mannequins, and humanoid robots. In this study, we show the possibility of eliciting a full-BOI over not one, but multiple artificial bodies concurrently. We demonstrate this by describing a system that allowed a participant to inhabit and fully control two different humanoid robots located in two distinct places and a virtual body in immersive virtual reality, using real-time full-body tracking and two-way audio communication, thereby giving them the illusion of ownership over each of them. We implemented this by allowing the participant be embodied in any one surrogate body at a given moment and letting them instantaneously switch between them. While the participant was embodied in one of the bodies, a proxy system would track the locations currently unoccupied and would control their remote representation in order to continue performing the tasks in those locations in a logical fashion. To test the efficacy of this system, an exploratory study was carried out with a fully functioning setup with three destinations and a simplified version of the proxy for use in a social interaction. The results indicate that the system was physically and psychologically comfortable and was rated highly by participants in terms of usability. Additionally, feelings of BOI and agency were reported, which were not influenced by the type of body representation. The results provide us with clues regarding BOI with humanoid robots of different dimensions, along with insight about self-localization and multilocation.

show abstract

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 99%

Multi-Destination Beaming: Apparently Being in Three Places at Once through Robotic and Virtual Embodiment

et al. 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…With a better understanding of these techniques, however, other usages have appeared, including the control of mobile robots by healthy subjects in various applications. The work by Kishore et al [28], targeting the control of a humanoid robot, is a representative example of the most common approach: the interaction is made through a screen, where all possible commands are associated with visual stimuli [52]. When the subject looks at a given command on the screen, the associated stimulus frequency is detected in the EEG signal and the command is triggered.…”

Section: State Of the Artmentioning

confidence: 99%

Electroencephalography as implicit communication channel for proximal interaction between humans and robot swarms

Luca¹,

Karim

Mondada

2016

Swarm Intell

View full text Add to dashboard Cite

Search and rescue, autonomous construction, and many other semi-autonomous multi-robot applications can benefit from proximal interactions between an operator and a swarm of robots. Most research on proximal interaction is based on explicit communication techniques such as gesture and speech. This study proposes a new implicit proximal communication technique to approach the problem of robot selection. We use electroencephalography (EEG) signals to select the robot at which the operator is looking. This is achieved using steady-state visually evoked potential (SSVEP), a repeatable neural response to a regularly blinking visual stimulus that varies predictively based on the blinking frequency. In our experiments, each robot was equipped with LEDs blinking at a different frequency, and the operator's SSVEP neural response was extracted from the EEG signal to detect and select the robot without requiring any conscious action by the user. This study systematically investigates several parameters affecting the SSVEP neural response: blinking frequency of the LED, distance between the robot and the operator, and color of the LED. Based on these parameters, we study two signal processing approaches and critically analyze their performance on 10 subjects controlling a set of physical robots. Our results show that despite numerous artifacts, it is possible to achieve a recognition rate higher than 85% on some subjects, while the average over the ten subjects was 75%.

show abstract

“…4 However, the latency of camera panning inherent in the mechanical systems can induce significant simulator sickness (Hals et al, 2014). Head tracking has also been proposed to directly control the drone navigation and rotations with attached fixed cameras; however, those systems also induced severe lag and motion sickness (Mirk and Hlavacs, 2015), even if a similar approach might work in other robotic scenarios without navigation (Kishore et al, 2014). Overall, latencies in panning can be overcome with digital panning of wide video content captured by wide-angle cameras.…”

mentioning

confidence: 99%

Stereoscopic First Person View System for Drone Navigation

Smolyanskiy

González-Franco

2017

Front. Robot. AI

View full text Add to dashboard Cite

Ground control of unmanned aerial vehicles (UAV) is a key to the advancement of this technology for commercial purposes. The need for reliable ground control arises in scenarios where human intervention is necessary, e.g. handover situations when autonomous systems fail. Manual flights are also needed for collecting diverse datasets to train deep neural network-based control systems. This axiom is even more prominent for the case of unmanned flying robots where there is no simple solution to capture optimal navigation footage. In such scenarios, improving the ground control and developing better autonomous systems are two sides of the same coin. To improve the ground control experience, and thus the quality of the footage, we propose to upgrade onboard teleoperation systems to a fully immersive setup that provides operators with a stereoscopic first person view (FPV) through a virtual reality (VR) head-mounted display. We tested users (n = 7) by asking them to fly our drone on the field. Test flights showed that operators flying our system can take off, fly, and land successfully while wearing VR headsets. In addition, we ran two experiments with prerecorded videos of the flights and walks to a wider set of participants (n = 69 and n = 20) to compare the proposed technology to the experience provided by current drone FPV solutions that only include monoscopic vision. Our immersive stereoscopic setup enables higher accuracy depth perception, which has clear implications for achieving better teleoperation and unmanned navigation. Our studies show comprehensive data on the impact of motion and simulator sickness in case of stereoscopic setup. We present the device specifications as well as the measures that improve teleoperation experience and reduce induced simulator sickness. Our approach provides higher perception fidelity during flights, which leads to a more precise better teleoperation and ultimately translates into better flight data for training deep UAV control policies.

show abstract

Comparison of SSVEP BCI and Eye Tracking for Controlling a Humanoid Robot in a Social Environment

Cited by 51 publications

References 28 publications

Multi-Destination Beaming: Apparently Being in Three Places at Once through Robotic and Virtual Embodiment

Multi-Destination Beaming: Apparently Being in Three Places at Once through Robotic and Virtual Embodiment

Electroencephalography as implicit communication channel for proximal interaction between humans and robot swarms

Stereoscopic First Person View System for Drone Navigation

Contact Info

Product

Resources

About