Eye-Gaze Controlled Wheelchair Based on Deep Learning

Abstract

In this paper, we design a technologically intelligent wheelchair with eye-movement control for patients with ALS in a natural environment. The system consists of an electric wheelchair, a vision system, a two-dimensional robotic arm, and a main control system. The smart wheelchair obtains the eye image of the controller through a monocular camera and uses deep learning and an attention mechanism to calculate the eye-movement direction. In addition, starting from the relationship between the trajectory of the joystick and the wheelchair speed, we establish a motion acceleration model of the smart wheelchair, which reduces the sudden acceleration of the smart wheelchair during rapid motion and improves the smoothness of the motion of the smart wheelchair. The lightweight eye-movement recognition model is transplanted into an embedded AI controller. The test results show that the accuracy of eye-movement direction recognition is 98.49%, the wheelchair movement speed is up to 1 m/s, and the movement trajectory is smooth, without sudden changes.

Research on Intelligent Wheelchair Multimode Human–Computer Interaction and Assisted Driving Technology

“…In terms of human-computer interaction between the occupant and the wheelchair, Xu et al designed an intelligent wheelchair with eye-movement control, which acquires the occupant's eye image through a camera and uses deep learning to determine the direction of eye movement, as well as to establish a motion acceleration model for the intelligent wheelchair to improve motion smoothness [15]. Cui et al proposed an intelligent wheelchair posture adjustment method based on action intent recognition, which adjusts the wheelchair posture by investigating the relationship between the force changes on the contact surface between the human body and the wheelchair and the action intent [16].…”

Smart Wheelchair Design for Elderly and Disabled People

“…The electronic brain of the system processes sensor information, automatically generates motor commands and display messages. The control module of the smart wheelchair can consist of a standard wheelchair joystick [2], voice recognition based control [3], facial expressions control [4], and even eye-gaze control [5] etc. The overall architecture of a wheelchair control system is shown in Figure 1.…”