Obstacle Recognition using Computer Vision and Convolutional Neural Networks for Powered Prosthetic Leg Applications

Novo-Torres, Luis; Ramirez-Paredes, Juan-Pablo; Villarreal, Dario J.

doi:10.1109/embc.2019.8857420

Cited by 17 publications

(25 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In comparison, the previous largest dataset contained approximately 402,000 images (Massalin et al, 2018). While most environment recognition systems have included fewer than 6 classes (Khademi and Simon, 2019; Krausz and Hargrove, 2015; Krausz et al, 2015; 2019; Laschowski et al, 2019b; Massalin et al, 2018; Novo-Torres et al, 2019; Varol and Massalin, 2016; Zhang et al, 2019b; 2019c; 2019d; 2020), the ExoNet database features a 12-class hierarchical labelling architecture. These differences have practical implications given that learning-based algorithms like deep convolutional neural networks require significant and diverse training images (LeCun et al, 2015).…”

Section: Discussionmentioning

confidence: 99%

“…The spatial resolution of the ExoNet images (1280×720) is considerably higher than previous efforts (e.g., 224×224 and 320×240). Poor image resolution has been attributed to decreased classification accuracy of human locomotion environments (Novo-Torres et al, 2019). Although higher resolution images can increase the computational and memory storage requirements, that being unfavourable for real-time mobile computing, research has been moving towards the development of efficient convolutional neural networks that require fewer operations (Tan and Le, 2020), therein enabling the processing of larger images for relatively similar computational power.…”

Section: Discussionmentioning

confidence: 99%

“…However, wearable vision-based systems can provide more detailed information about the field-of-view and detect physical obstacles in peripheral locations. Most environment recognition systems have included either RGB cameras (Da Silva et al, 2020; Diaz et al, 2018; Khademi and Simon, 2019; Krausz and Hargrove, 2015; Laschowski et al, 2019b; Novo-Torres et al, 2019; Zhong et al, 2020) or 3D depth cameras (Hu et al, 2018; Krausz et al, 2015; 2019; Massalin et al, 2018; Varol and Massalin, 2016; Zhang et al, 2019b; 2019c; 2019d).…”

Section: Introductionmentioning

confidence: 99%

“…For image classification, researchers have used learning-based algorithms like support vector machines (Massalin et al, 2018; Varol and Massalin, 2016) and deep convolutional neural networks (Khademi and Simon, 2019; Laschowski et al, 2019b; Novo-Torres et al, 2019; Rai and Rombokas, 2018; Zhang et al, 2019b; 2019c; 2019d; Zhong et al, 2020). Although convolutional neural networks typically outperform support vector machines for image classification (LeCun et al, 2015), deep learning requires significant and diverse training images to prevent overfitting and promote generalization.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Laschowski

McNally

Wong

et al. 2020

Preprint

View full text Add to dashboard Cite

Advances in computer vision and artificial intelligence are allowing researchers to develop environment recognition systems for powered lower-limb exoskeletons and prostheses. However, small-scale and private training datasets have impeded the widespread development and dissemination of image classification algorithms for classifying human walking environments. To address these limitations, we developed ExoNet - the first open-source, large-scale hierarchical database of high-resolution wearable camera images of human locomotion environments. Unparalleled in scale and diversity, ExoNet contains over 5.6 million RGB images of different indoor and outdoor real-world walking environments, which were collected using a lightweight wearable camera system throughout the summer, fall, and winter seasons. Approximately 923,000 images in ExoNet were human-annotated using a 12-class hierarchical labelling architecture. Available publicly through IEEE DataPort, ExoNet offers an unprecedented communal platform to train, develop, and compare next-generation image classification algorithms for human locomotion environment recognition. Besides the control of powered lower-limb exoskeletons and prostheses, applications of ExoNet could extend to humanoids and autonomous legged robots.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ExoNet Database: Wearable Camera Images of Human Locomotion Environments

Laschowski

McNally

Wong

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Additionally, we presented work to use depth sensing for gait segmentation, in addition to inertial sensors, to improve timing of steps, which ultimately should improve forward prediction as well [15]. Recently, several other research groups have presented preliminary results for the development of other predictive systems for powered prostheses or exoskeletons that use environmental sensors, including [11,16,17,18,19,20,21,22,23,24,25]. These works are important and show the utility of using environmental based sensors for improving high-level control of assistive devices.…”

Section: Introductionmentioning

confidence: 99%

Subject- and Environment-Based Sensor Variability for Wearable Lower-Limb Assistive Devices

Krausz

Hargrove

2019

Sensors

View full text Add to dashboard Cite

Significant research effort has gone towards the development of powered lower limb prostheses that control power during gait. These devices use forward prediction based on electromyography (EMG), kinetics and kinematics to command the prosthesis which locomotion activity is desired. Unfortunately these predictions can have substantial errors, which can potentially lead to trips or falls. It is hypothesized that one reason for the significant prediction errors in the current control systems for powered lower-limb prostheses is due to the inter- and intra-subject variability of the data sources used for prediction. Environmental data, recorded from a depth sensor worn on a belt, should have less variability across trials and subjects as compared to kinetics, kinematics and EMG data, and thus its addition is proposed. The variability of each data source was analyzed, once normalized, to determine the intra-activity and intra-subject variability for each sensor modality. Then measures of separability, repeatability, clustering and overall desirability were computed. Results showed that combining Vision, EMG, IMU (inertial measurement unit), and Goniometer features yielded the best separability, repeatability, clustering and desirability across subjects and activities. This will likely be useful for future application in a forward predictor, which will incorporate Vision-based environmental data into a forward predictor for powered lower-limb prosthesis and exoskeletons.

show abstract