Environmental Context Prediction for Lower Limb Prostheses With Uncertainty Quantification

Zhong, Boxuan; Silva, Rafael Luiz da; Li, Minhan; Huang, He; Lobatón, Edgar

doi:10.1109/tase.2020.2993399

Cited by 54 publications

(41 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Studies have shown that supplementing an automated high-level controller with terrain information improved the classification accuracies and decision times compared to excluding the environmental context [4]- [5]. Common wearables used for environment sensing include radar detectors [6], laser rangefinders [4]- [5], [7], RGB cameras [8]- [13], and 3D depth cameras [14]- [19] (Fig. 1).…”

Section: Introductionmentioning

confidence: 99%

Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons

Laschowski

McNally

Wong

et al. 2021

Preprint

View full text Add to dashboard Cite

Robotic exoskeletons require human control and decision making to switch between different locomotion modes, which can be inconvenient and cognitively demanding. To support the development of automated locomotion mode recognition systems (i.e., high-level controllers), we designed an environment recognition system using computer vision and deep learning. We collected over 5.6 million images of indoor and outdoor real-world walking environments using a wearable camera system, of which ~923,000 images were annotated using a 12-class hierarchical labelling architecture (called the ExoNet database). We then trained and tested the EfficientNetB0 convolutional neural network, designed for efficiency using neural architecture search, to predict the different walking environments. Our environment recognition system achieved ~73% image classification accuracy. While these preliminary results benchmark EfficientNetB0 on the ExoNet database, further research is needed to compare different image classification algorithms to develop an accurate and real-time environment-adaptive locomotion mode recognition system for robotic exoskeleton control.

show abstract

Section: Introductionmentioning

confidence: 99%

Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons

Laschowski

McNally

Wong

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, vision-based systems can provide more detailed information about the field-of-view and detect physical obstacles in peripheral locations. Most environment recognition systems have included either RGB cameras (Krausz and Hargrove, 2015 ; Diaz et al, 2018 ; Khademi and Simon, 2019 ; Laschowski et al, 2019b ; Novo-Torres et al, 2019 ; Da Silva et al, 2020 ; Zhong et al, 2020 ) or 3D depth cameras (Krausz et al, 2015 , 2019 ; Varol and Massalin, 2016 ; Hu et al, 2018 ; Massalin et al, 2018 ; Zhang et al, 2019b , c , d ).…”

Section: Introductionmentioning

confidence: 99%

“…For image classification, researchers have used learning-based algorithms like support vector machines (Varol and Massalin, 2016 ; Massalin et al, 2018 ) and deep convolutional neural networks (Rai and Rombokas, 2018 ; Khademi and Simon, 2019 ; Laschowski et al, 2019b ; Novo-Torres et al, 2019 ; Zhang et al, 2019b , c , d ; Zhong et al, 2020 ). Although convolutional neural networks typically outperform support vector machines for image classification (LeCun et al, 2015 ), deep learning requires significant and diverse training images to prevent overfitting and promote generalization.…”

Section: Introductionmentioning

confidence: 99%

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

et al. 2020

View full text Add to dashboard Cite

“…However, wearable vision-based systems can provide more detailed information about the field-of-view and detect physical obstacles in peripheral locations. Most environment recognition systems have included either RGB cameras (Da Silva et al, 2020; Diaz et al, 2018; Khademi and Simon, 2019; Krausz and Hargrove, 2015; Laschowski et al, 2019b; Novo-Torres et al, 2019; Zhong et al, 2020) or 3D depth cameras (Hu et al, 2018; Krausz et al, 2015; 2019; Massalin et al, 2018; Varol and Massalin, 2016; Zhang et al, 2019b; 2019c; 2019d).…”

Section: Introductionmentioning

confidence: 99%

“…For image classification, researchers have used learning-based algorithms like support vector machines (Massalin et al, 2018; Varol and Massalin, 2016) and deep convolutional neural networks (Khademi and Simon, 2019; Laschowski et al, 2019b; Novo-Torres et al, 2019; Rai and Rombokas, 2018; Zhang et al, 2019b; 2019c; 2019d; Zhong et al, 2020). Although convolutional neural networks typically outperform support vector machines for image classification (LeCun et al, 2015), deep learning requires significant and diverse training images to prevent overfitting and promote generalization.…”

Section: Introductionmentioning

confidence: 99%

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Laschowski

McNally

Wong

et al. 2020

Preprint

View full text Add to dashboard Cite

Advances in computer vision and artificial intelligence are allowing researchers to develop environment recognition systems for powered lower-limb exoskeletons and prostheses. However, small-scale and private training datasets have impeded the widespread development and dissemination of image classification algorithms for classifying human walking environments. To address these limitations, we developed ExoNet - the first open-source, large-scale hierarchical database of high-resolution wearable camera images of human locomotion environments. Unparalleled in scale and diversity, ExoNet contains over 5.6 million RGB images of different indoor and outdoor real-world walking environments, which were collected using a lightweight wearable camera system throughout the summer, fall, and winter seasons. Approximately 923,000 images in ExoNet were human-annotated using a 12-class hierarchical labelling architecture. Available publicly through IEEE DataPort, ExoNet offers an unprecedented communal platform to train, develop, and compare next-generation image classification algorithms for human locomotion environment recognition. Besides the control of powered lower-limb exoskeletons and prostheses, applications of ExoNet could extend to humanoids and autonomous legged robots.

show abstract

Environmental Context Prediction for Lower Limb Prostheses With Uncertainty Quantification

Cited by 54 publications

References 21 publications

Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons

Computer Vision and Deep Learning for Environment-Adaptive Control of Robotic Lower-Limb Exoskeletons

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Contact Info

Product

Resources

About