Evaluation of a Visual Localization System for Environment Awareness in Assistive Devices

Rai, Vijeth

doi:10.1109/embc.2018.8513442

Cited by 6 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One subject was instrumented with a lightweight wearable smartphone camera system (iPhone XS Max); photograph shown in Figure 1A . Unlike limb-mounted systems (Zhang et al, 2011 , 2019b , c ; Varol and Massalin, 2016 ; Diaz et al, 2018 ; Hu et al, 2018 ; Kleiner et al, 2018 ; Massalin et al, 2018 ; Rai and Rombokas, 2018 ; Da Silva et al, 2020 ), chest-mounting can provide more stable video recording and allow users to wear pants and long dresses without obstructing the sampled field-of-view. The chest-mount height was ~1.3 m from the ground when the participant stood upright.…”

Section: Methodsmentioning

confidence: 99%

“…For image classification, researchers have used learning-based algorithms like support vector machines (Varol and Massalin, 2016 ; Massalin et al, 2018 ) and deep convolutional neural networks (Rai and Rombokas, 2018 ; Khademi and Simon, 2019 ; Laschowski et al, 2019b ; Novo-Torres et al, 2019 ; Zhang et al, 2019b , c , d ; Zhong et al, 2020 ). Although convolutional neural networks typically outperform support vector machines for image classification (LeCun et al, 2015 ), deep learning requires significant and diverse training images to prevent overfitting and promote generalization.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

et al. 2020

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

et al. 2020

View full text Add to dashboard Cite

“…One subject was instrumented with a lightweight wearable smartphone camera system (iPhone XS Max); photograph shown in Figure 1A. Unlike limb-mounted systems (Da Silva et al, 2020; Diaz et al, 2018; Hu et al, 2018; Kleiner et al, 2018; Massalin et al, 2018; Rai and Rombokas, 2018; Varol and Massalin, 2016; Zhang et al, 2011; 2019b; 2019c), chest-mounting can provide more stable video recording and allow users to wear pants and long dresses without obstructing the sampled field-of-view. The chest-mount height was approximately 1.3 m from the ground when the participant stood upright.…”

Section: Methodsmentioning

confidence: 99%

“…For image classification, researchers have used learning-based algorithms like support vector machines (Massalin et al, 2018; Varol and Massalin, 2016) and deep convolutional neural networks (Khademi and Simon, 2019; Laschowski et al, 2019b; Novo-Torres et al, 2019; Rai and Rombokas, 2018; Zhang et al, 2019b; 2019c; 2019d; Zhong et al, 2020). Although convolutional neural networks typically outperform support vector machines for image classification (LeCun et al, 2015), deep learning requires significant and diverse training images to prevent overfitting and promote generalization.…”

Section: Introductionmentioning

confidence: 99%

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Laschowski

McNally

Wong

et al. 2020

Preprint

View full text Add to dashboard Cite

Advances in computer vision and artificial intelligence are allowing researchers to develop environment recognition systems for powered lower-limb exoskeletons and prostheses. However, small-scale and private training datasets have impeded the widespread development and dissemination of image classification algorithms for classifying human walking environments. To address these limitations, we developed ExoNet - the first open-source, large-scale hierarchical database of high-resolution wearable camera images of human locomotion environments. Unparalleled in scale and diversity, ExoNet contains over 5.6 million RGB images of different indoor and outdoor real-world walking environments, which were collected using a lightweight wearable camera system throughout the summer, fall, and winter seasons. Approximately 923,000 images in ExoNet were human-annotated using a 12-class hierarchical labelling architecture. Available publicly through IEEE DataPort, ExoNet offers an unprecedented communal platform to train, develop, and compare next-generation image classification algorithms for human locomotion environment recognition. Besides the control of powered lower-limb exoskeletons and prostheses, applications of ExoNet could extend to humanoids and autonomous legged robots.

show abstract

“…The latest generation of environment recognition systems has used convolutional neural networks (CNNs) for image classification (Khademi and Simon, 2019;Laschowski et al, 2019b;2021b;Novo-Torres et al, 2019;Rai and Rombokas, 2018;Zhang et al, 2019b;2019c;2019d;2020;Zhong et al, 2020) (see Table 3 and Figure 4). One of the earliest publications came from Laschowski and colleagues (2019b), who designed and trained a 10-layer convolutional neural network using five-fold cross-validation, which differentiated between three environment classes with 94.9% classification accuracy.…”

Section: Literature Reviewmentioning

confidence: 99%

Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks

Laschowski

McNally

Wong

et al. 2021

Preprint

View full text Add to dashboard Cite

Robotic leg prostheses and exoskeletons can provide powered locomotor assistance to older adults and/or persons with physical disabilities. However, the current locomotion mode recognition systems being developed for intelligent high-level control and decision-making use mechanical, inertial, and/or neuromuscular data, which inherently have limited prediction horizons (i.e., analogous to walking blindfolded). Inspired by the human vision-locomotor control system, we designed and evaluated an advanced environment classification system that uses computer vision and deep learning to forward predict the oncoming walking environments prior to physical interaction, therein allowing for more accurate and robust locomotion mode transitions. In this study, we first reviewed the development of the ExoNet database – the largest and most diverse open-source dataset of wearable camera images of indoor and outdoor real-world walking environments, which were annotated using a hierarchical labelling architecture. We then trained and tested over a dozen state-of-the-art deep convolutional neural networks (CNNs) on the ExoNet database for large-scale image classification of the walking environments, including: EfficientNetB0, InceptionV3, MobileNet, MobileNetV2, VGG16, VGG19, Xception, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, and DenseNet201. Lastly, we quantitatively compared the benchmarked CNN architectures and their environment classification predictions using an operational metric called NetScore, which balances the image classification accuracy with the computational and memory storage requirements (i.e., important for onboard real-time inference). Although we designed this environment classification system to support the development of next-generation environment-adaptive locomotor control systems for robotic prostheses and exoskeletons, applications could extend to humanoids, autonomous legged robots, powered wheelchairs, and assistive devices for persons with visual impairments.

show abstract

Evaluation of a Visual Localization System for Environment Awareness in Assistive Devices

Cited by 6 publications

References 17 publications

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Environment Classification for Robotic Leg Prostheses and Exoskeletons using Deep Convolutional Neural Networks

Contact Info

Product

Resources

About