Motor learning affects car-to-driver handover in automated vehicles

Russell, Holly E. B.; Harbott, Lene K.; Nisky, Ilana; Pan, Selina; Okamura, Allison M.; Gerdes, J. Christian

doi:10.1126/scirobotics.aah5682

Cited by 105 publications

(84 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This approach can improve teleoperation and handover scenarios (Russell et al, 2016), can simplify autonomous navigation (compared to monoscopic systems), and help in capturing better video footage for training deep control policies (Artale et al, 2013). Our results show that in the real-time condition, with our system, users can fly and land without problems.…”

Section: Discussionmentioning

confidence: 83%

“…Furthermore, the combination of a human FPV drone control with automatic lowlevel obstacle avoidance is superior to fully autonomous systems at the time of writing. In fact, current regulations are required to contemplate handover situations (Russell et al, 2016). While the human operator is responsible for higher level navigation guidance with current regulations, the low-level automatic obstacle avoidance could decrease operator's cognitive load and increase the security of the system.…”

Section: Discussionmentioning

confidence: 99%

“…Both researchers and regulators are pointing at mixed scenarios in which control of autonomous flying robots creates handover situations where humans must regain control under specific circumstances (Russell et al, 2016). UAV fly at high speeds and can be dangerous.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Stereoscopic First Person View System for Drone Navigation

Smolyanskiy

González-Franco

2017

Front. Robot. AI

View full text Add to dashboard Cite

Ground control of unmanned aerial vehicles (UAV) is a key to the advancement of this technology for commercial purposes. The need for reliable ground control arises in scenarios where human intervention is necessary, e.g. handover situations when autonomous systems fail. Manual flights are also needed for collecting diverse datasets to train deep neural network-based control systems. This axiom is even more prominent for the case of unmanned flying robots where there is no simple solution to capture optimal navigation footage. In such scenarios, improving the ground control and developing better autonomous systems are two sides of the same coin. To improve the ground control experience, and thus the quality of the footage, we propose to upgrade onboard teleoperation systems to a fully immersive setup that provides operators with a stereoscopic first person view (FPV) through a virtual reality (VR) head-mounted display. We tested users (n = 7) by asking them to fly our drone on the field. Test flights showed that operators flying our system can take off, fly, and land successfully while wearing VR headsets. In addition, we ran two experiments with prerecorded videos of the flights and walks to a wider set of participants (n = 69 and n = 20) to compare the proposed technology to the experience provided by current drone FPV solutions that only include monoscopic vision. Our immersive stereoscopic setup enables higher accuracy depth perception, which has clear implications for achieving better teleoperation and unmanned navigation. Our studies show comprehensive data on the impact of motion and simulator sickness in case of stereoscopic setup. We present the device specifications as well as the measures that improve teleoperation experience and reduce induced simulator sickness. Our approach provides higher perception fidelity during flights, which leads to a more precise better teleoperation and ultimately translates into better flight data for training deep UAV control policies.

show abstract

Section: Discussionmentioning

confidence: 83%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Stereoscopic First Person View System for Drone Navigation

Smolyanskiy

González-Franco

2017

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…The metric of time to handwheel peak includes feedback components, but, importantly, also represents the feed forward component of driver steering control (Russell et al, 2016). Our results suggest that the driver plans to start their lane-change maneuver earlier under the incongruent steering condition, as a strategy to gain more time for any necessary steering corrections.…”

Section: Discussionmentioning

confidence: 99%

“…The following variables (described previously (Russell et al, 2016); see also Supporting Information, Figure 1) were recorded at 60 Hz, quantified for each trial and averaged during the time indicated for each turning and straight portions of the course.…”

Section: Methodsmentioning

confidence: 99%

Mind over motor mapping: Driver response to changing vehicle dynamics

Bruno

Baker

Gundran

et al. 2018

Human Brain Mapping

Self Cite

View full text Add to dashboard Cite

Improvements in vehicle safety require understanding of the neural systems that support the complex, dynamic task of real-world driving. We used functional near infrared spectroscopy (fNIRS) and pupilometry to quantify cortical and physiological responses during a realistic, simulated driving task in which vehicle dynamics were manipulated. Our results elucidate compensatory changes in driver behavior in response to changes in vehicle handling. We also describe associated neural and physiological responses under different levels of mental workload. The increased cortical activation we observed during the late phase of the experiment may indicate motor learning in prefrontal-parietal networks. Finally, relationships among cortical activation, steering control, and individual personality traits suggest that individual brain states and traits may be useful in predicting a driver's response to changes in vehicle dynamics. Results such as these will be useful for informing the design of automated safety systems that facilitate safe and supportive driver-car communication.

show abstract

Human–Machine Collaboration for Automated Driving Using an Intelligent Two‐Phase Haptic Interface

Xing

et al. 2021

Advanced Intelligent Systems

View full text Add to dashboard Cite

Prior to realizing fully autonomous driving, human intervention is periodically required to guarantee vehicle safety. This poses a new challenge in humanmachine interaction, particularly during the control authority transition from automated functionality to a human driver. Herein, this challenge is addressed by proposing an intelligent haptic interface based on a newly developed two-phase human-machine interaction model. The intelligent haptic torque is applied to the steering wheel and switches its functionality between predictive guidance and haptic assistance according to the varying state and control ability of human drivers. This helps drivers gradually resume manual control during takeover. The developed approach is validated by conducting vehicle experiments with 26 participants. The results suggest that the proposed method effectively enhances the driving state recovery and control performance of human drivers during takeover compared with an existing approach. Thus, this new method further improves the safety and smoothness of human-machine interaction in automated vehicles.

show abstract

Motor learning affects car-to-driver handover in automated vehicles

Abstract: A human driver requires a period of motor adaptation to resume normal steering after taking control of an automated vehicle.

Cited by 105 publications

References 34 publications

Stereoscopic First Person View System for Drone Navigation

Stereoscopic First Person View System for Drone Navigation

Mind over motor mapping: Driver response to changing vehicle dynamics

Human–Machine Collaboration for Automated Driving Using an Intelligent Two‐Phase Haptic Interface

Contact Info

Product

Resources

About