Integral Real-time Locomotion Mode Recognition Based on GA-CNN for Lower Limb Exoskeleton

Abstract

The wearable lower limb exoskeleton is a typical human-in-loop human–robot coupled system, which conducts natural and close cooperation with the human by recognizing human locomotion timely. Requiring subject-specific training is the main challenge of the existing approaches, and most methods have the problem of insufficient recognition. This paper proposes an integral subject-adaptive real-time Locomotion Mode Recognition (LMR) method based on GA-CNN for a lower limb exoskeleton system. The LMR method is a combination of Convolutional Neural Networks (CNN) and Genetic Algorithm (GA)-based multi-sensor information selection. To improve network performance, the hyper-parameters are optimized by Bayesian optimization. An exoskeleton prototype system with multi-type sensors and novel sensing-shoes is used to verify the proposed method. Twelve locomotion modes, which composed an integral locomotion system for the daily application of the exoskeleton, can be recognized by the proposed method. According to a series of experiments, the recognizer shows strong comprehensive abilities including high accuracy, low delay, and sufficient adaption to different subjects.

A Review on Locomotion Mode Recognition and Prediction When Using Active Orthoses and Exoskeletons

“…Based on Table 2 , the most investigated LMs are dynamic tasks only [ 2 , 4 , 5 , 8 , 10 , 13 , 30 , 31 , 32 ] (N = 11) and a combination between dynamic and static tasks [ 3 , 6 , 9 , 11 , 12 , 34 , 35 ] (N = 7). Among the dynamic tasks, six studies [ 4 , 13 , 31 , 33 , 34 , 35 ] classified continuous dynamic tasks, whereas only one study [ 2 ] focused on transitions. Thirteen studies [ 3 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 30 , 32 , 34 , 35 ] explored continuous and discrete dynamic tasks together.…”

“…Among the dynamic tasks, six studies [ 4 , 13 , 31 , 33 , 34 , 35 ] classified continuous dynamic tasks, whereas only one study [ 2 ] focused on transitions. Thirteen studies [ 3 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 30 , 32 , 34 , 35 ] explored continuous and discrete dynamic tasks together.…”

“…The ST task was the most investigated static LM [ 3 , 6 , 7 , 9 , 11 , 12 , 34 , 35 ] (N = 8), whereas the SIT task was explored in three studies [ 6 , 7 , 35 ]. Three studies [ 6 , 7 , 35 ] considered the identification of both ST and SIT tasks.…”

“…Concerning the dynamic LMs, the LW, SA, and SD tasks were the continuous dynamic tasks typically addressed, being explored in 17 studies [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 30 , 31 , 32 , 33 , 34 , 35 ], while the RA and RD tasks were investigated together with the LW, SA, and SD tasks in nine studies [ 4 , 8 , 11 , 12 , 13 , 30 , 31 , 32 , 35 ]. The LW-SA, SA-LW, LW-SD, and SD-LW transitions were addressed in eight studies [ 5 , 6 , 7 , 8 , 9 , 10 , 30 , 32 ], being the most investigated discrete dynamic tasks.…”

“…The LW-SA, SA-LW, LW-SD, and SD-LW transitions were addressed in eight studies [ 5 , 6 , 7 , 8 , 9 , 10 , 30 , 32 ], being the most investigated discrete dynamic tasks. The LW-RA, RA-LW, LW-RD, and RD-LW tasks were explored in four studies [ 8 , 9 , 30 , 32 ], the SIT-ST and ST-SIT in four studies [ 2 , 6 , 7 , 35 ], the ST-LW and LW-ST in two studies [ 6 , 7 ], and the ST-SA and SA-ST in one study [ 6 ].…”