Sensory Cues Guided Rehabilitation Robotic Walker Realized by Depth Image-Based Gait Analysis

Automation enhancement and accuracy investigation of a portable single‐camera gait analysis system

“…Future work consists of extending the system to a stereo 2D-camera system or a single depth sensing device, and investigation of the error with a focus on marker trajectories. Our future work will be focused on further improvement of performance and potential measurement capability of more gait parameters using a stereo 2D-camera system or a single depth sensing device [2,[13][14][15][16][17][18][19]] with a high frame rate, without sacrificing portability, to remove the parallax error, and leverage the 3D information for quantifying a larger number of gait parameters such as hip, knee, and ankle angles in both the sagittal and frontal planes, and pelvis tilt, calculating temporal-spatial parameters (step length and width, stride length, step time, cadence and step length symmetry), gait speed and measuring sagittal / frontal plane knee motion, but at an increased processing complexity. Furthermore, neural-networkbased methods are what we could consider in future research that involve evaluations with public availability of large labelled datasets of walking trials.…”

“…Related range sensor and home-video based systems, which cost about £700, such as [9], [10] and [11], that build on the work of [12], with Pro-Trainer motion analysis software (Sports Motion, Inc., Cardiff, CA), offer gait analysis outside the gait laboratory, e.g., in local clinics and at homes. Similar to other range sensor and home-video based gait analysis systems [2,[13][14][15][16][17][18][19][20][21] and Inertial Measurement Unit (IMU) based gait analysis systems [22][23][24][25][26], the gait parameters obtained after data processing can be sent to physiatrists for clinical consultation, indicating the potential for tele-rehabilitation [27][28][29][30][31]. It is shown in [32] that a 2D video tracker software provides similar accuracy to VICON 3D system for knee angle measurement but not for measurement of the ankle angle over time.…”

“…Our system addresses some of the drawbacks of related range sensor and home-video based systems [2,9,11,[13][14][15][16][17][18][19]41] and IMU systems [22][23][24][25][26] namely: (1) Unlike [9], there are no colour restrictions on the background or the participant's clothing; (2) In contrast to Soda et al [9], which is validated on only one healthy volunteer with one walking trial with no gold standard benchmark, we validate our proposed system's knee angle against the gold standard VICON MX Giganet 6xT40 and 6xT160 (VICON Motion Systems Ltd., Oxford, UK, approximately £250,000) optical motion analysis system (the same gold standard as used by [11]). (3) Unlike systems of [11] and Pro-Trainer and Siliconcoach (Siliconcoach Ltd., Dunedin, New Zealand) as used by [42] and [43] that require significant manual effort, our system autonomously tracks the markers attached to the joints and calculates the knee angle; the only operational effort required is for marker-template selection for tracking initialization which is done via a user-friendly graphical user interface (GUI).…”

“…Our system addresses some of the drawbacks of related range sensor and home-video based systems [2,9,11,[13][14][15][16][17][18][19]41] and IMU systems [22][23][24][25][26] namely: (i) unlike [9], there are no colour restrictions on the background or the participant's clothing; (ii) in contrast to [9], which is validated on only one healthy volunteer with one walking trial with no gold standard benchmark, we validate our proposed system's knee angle against the gold standard VICON MX Giganet 6xT40 and 6xT160 (VICON Motion Systems Ltd., Oxford, UK, approximately £ 250,000) optical motion analysis system (the same gold standard as used by Ugbolue et al [11]); (iii) unlike systems of [11] and Pro-Trainer and Siliconcoach (Siliconcoach Ltd., Dunedin, New Zealand) as used by the authors in [42,43] that require significant manual effort, our system autonomously tracks the markers attached to the joints and calculates the knee angle; the only operational effort required is for marker-template selection for tracking initialisation which is done via a user-friendly graphical user interface (GUI); (iv) unlike the passive marker system [41] that is only validated on one side of the body without any benchmarking systems, our system is validated on both sides of the body with a gold standard VICON optical motion analysis system; (v) 3D Kinect range sensor-based systems [13-18, 20, 21] cannot reliably capture relatively fast body motion, since Kinect operates at only 30 frames per second (fps), whereas our system operates at 210 fps; (vi) like other range sensor and home-video based systems, our system is non-intrusive to the participants, which is in contrast to state-of-the-art IMU gait analysis systems [22][23][24][25][26]. However, with only a 2D camera in our gait analysis system, its drawback lies in the following two aspects: (i) estimation of the human joint locations using our system is less accurate compared to 3D Kinect-based range sensor systems, and (ii)...…”

HD: Efficient Hand Detection and Tracking

“…Medical applications involving the monitoring of human interaction with robots (HRI) [12] can embed hand detection, e.g. for functional rehabilitation [13] such as exercises of patient's interaction with objects s/he grasps [14] to improve stroke recovery [15].…”