Development of Distributed Control System for Vision-Based Myoelectric Prosthetic Hand

He, Yong; Shima, Ryusei; Fukuda, Osamu; Bu, Nan; Yamaguchi, Nobuhiko; Okumura, Hiroshi

doi:10.1109/access.2019.2911968

Cited by 18 publications

(17 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our research group has also been developing a visionbased prosthetic hand based on deep learning technology [35] [36] [37] [38]. In [39], we designed a prosthetic hand control method that can determine the grasping target and motion according to the spatial and temporal relationship between the prosthetic hand and the objects, such as distance, position, and gazing time.…”

Section: Related Workmentioning

confidence: 99%

“…This input-output relationship is extremely complex and nonlinear, but humans naturally FIGURE 1. Vision-based prosthetic hand [38] obtain the input-output map through learning with a single central nerve network. Referring to the excellent mechanism of human beings, we attempt to implement a single deep learning network in the system and model nonlinear mapping between images, sEMG signals, and prosthetic hand motions in an end-to-end training manner.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bimodal Control of a Vision-Based Myoelectric Hand

Fukuda

Sakaguchi

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a novel control scheme for a vision-based prosthetic hand. To realize complex and flexible human-like hand movements, the proposed method fuses bimodal information. Combining information from surface EMG signals with object information from a vision sensor, the system can select an appropriate hand motion. The training/recognition using both sEMG signals and object images can be performed with a single deep neural network in an end-to-end manner. The bimodal sensor information enables the system to recognize the operator's intended motion with higher accuracy than that of the conventional method using only sEMG signals. In addition, the generalization ability of the network is improved, so motion recognition robustness is enhanced against abnormal data that include partly noisy or missing samples. To verify the validity of the proposed approach, we prepared a dataset that contains the sEMG signals and the object images for 10 types of grasping motions. Three kinds of experiments were conducted: comparison of the proposed method with the conventional method, examination of the recognition robustness against partly noisy or missing samples, and challenges to recognize hand motions based on raw sEMG signals. The results revealed that the proposed bimodal network achieved considerably high recognition performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Bimodal Control of a Vision-Based Myoelectric Hand

Fukuda

Sakaguchi

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…5. It uses the same architecture as used in [17]. The network first extracts features from an entire image using the convolutional layers (backbone); then, the features are used for coordinate regression and class classification [30].…”

Section: A Target Object Selectionmentioning

confidence: 99%

Vision-Based Assistance for Myoelectric Hand Control

Kubozono

Fukuda

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Conventional control systems for prosthetic hands use myoelectric signals as an interface, but it is impossible to realize complex and flexible human hand movements with only myoelectric signals. A promising control scheme for prosthetic hands uses computer vision to assist in grasping objects. It features an imaging sensor, and the control system is capable of recognizing an object placed in the environment. Then, a gripping pattern can be selected from some predefined candidates according the recognized object. However, previous studies assumed that only one object exists in the environment. If there are multiple target objects in the environment, the hand could become confused in attempting to find the target object. This study addresses this problem and proposes a method to determine the target object from multiple objects. The proposed method is able to determine the target object by estimating the positional relationship between the artificial hand and the objects, as well as the motion of the hand. To verify the validity and effectiveness, we implemented the proposed method in a vision-based prosthetic hands control system and conducted pick-and-place experiments. The experiments confirm that the proposed method can accurately estimate the target object in accordance with the user's intention.

show abstract

“…The control system follows a simple rule that the object which is closest to the center of the image is considered as the target object. This rule is also used in study [4]. Another thing that needs to be concerned is that sometimes the regression of the object location is not quite accurate, and some parts of the object may not be included in the cropped object image.…”

Section: A Object Recognitionmentioning

confidence: 99%

“…Eyes then coordinate the hand to properly grasp the object. Inspired by the way that human grasps objects, many studies integrate cameras in the prosthetic hand control system [1], [2], [3], [4], [5], [6], [7], [8]. Such systems accept images as input and extract necessary information (e.g.…”

Section: Introductionmentioning

confidence: 99%

Noble Control Scheme for Prosthetic Hands through Spatial Understanding

He¹,

Fukuda²,

Yamaguchi³

et al. 2020

IJACSA

Self Cite

View full text Add to dashboard Cite

A noble control scheme for prosthetic hands through spatial understanding is proposed. The proposed control scheme features an imaging sensor and an inertial measurement unit (IMU) sensor, which makes prosthetic hands capable of visual and motion sensing. The imaging sensor captures the scene where the user is willing to grasp the object. The control system recognizes the target object, extracts its surface features and estimates its pose from the captured images. Then the spatial relationship is constructed between the hand and the target object. With the help of IMU sensors, the relationship can be tracked and kept wherever the prosthetic hand moves even the object is out of the view range of the camera. To interact with the user, this process is visualized using augmented reality (AR) technology. A test platform based on the proposed control scheme is developed and a case study is performed with the platform.

show abstract

Development of Distributed Control System for Vision-Based Myoelectric Prosthetic Hand

Cited by 18 publications

References 17 publications

Bimodal Control of a Vision-Based Myoelectric Hand

Bimodal Control of a Vision-Based Myoelectric Hand

Vision-Based Assistance for Myoelectric Hand Control

Noble Control Scheme for Prosthetic Hands through Spatial Understanding

Contact Info

Product

Resources

About