Attitude control for astronaut assisted robot in the space station

IEEE Trans. Ind. Electron.

et al. 2019

Self Cite

109

In this study, a parallel network based on hand detection and body pose estimation is proposed to detect and distinguish human's right and left hands. The network is employed to human-robot interaction (HRI) based on hand gestures. This method fully uses hand feature information and hand information in the human body structure. One channel in the network uses a ResNet-Inception-Single Shot MultiBox Detector to extract hand feature information for human's hand detection. The other channel estimates human body pose first and then estimates the positions of the left and right hands using the forward kinematic tree of the human skeleton structure. Thereafter, the results of the two channels are fused. In the fusion module, the human body structure can be utilized to correct hand detection results and distinguish between the right and left hands. Experimental results verify that the parallel deep neural network can effectively improve the accuracy of hand detection and distinguish between the right and left hands effectively. This method is also used for the hand gesturebased interaction between astronauts and an astronaut assistant robot. Our method can be suitably used in this HRI system.

Section: Dual-hand Detection Based On Ri-ssd and Body Pose Estimationmentioning

confidence: 99%

Dual-Hand Detection for Human–Robot Interaction by a Parallel Network Based on Hand Detection and Body Pose Estimation

IEEE Trans. Ind. Electron.

et al. 2019

Self Cite

109

“…Security and reliability play important roles in space HRI tasks. This chapter is aimed at a bionic manipulator on an astronaut assistant robot (AAR; Gao et al, 2017b; Liu et al, 2016a), using hand gestures to control and operate it.…”

Section: Gesture‐based Space Hri Systemmentioning

confidence: 99%

Hand gesture recognition using multimodal data fusion and multiscale parallel convolutional neural network for human–robot interaction

2020

Expert Systems

Self Cite

Hand gesture recognition plays an important role in human-robot interaction. The accuracy and reliability of hand gesture recognition are the keys to gesture-based human-robot interaction tasks. To solve this problem, a method based on multimodal data fusion and multiscale parallel convolutional neural network (CNN) is proposed in this paper to improve the accuracy and reliability of hand gesture recognition. First of all, data fusion is conducted on the sEMG signal, the RGB image, and the depth image of hand gestures. Then, the fused images are generated to two different scale images by downsampling, which are respectively input into two subnetworks of the parallel CNN to obtain two hand gesture recognition results. After that, hand gesture recognition results of the parallel CNN are combined to obtain the final hand gesture recognition result. Finally, experiments are carried out on a self-made database containing 10 common hand gestures, which verify the effectiveness and superiority of the proposed method for hand gesture recognition. In addition, the proposed method is applied to a seven-degree-of-freedom bionic manipulator to achieve robotic manipulation with hand gestures.

“…While in the astronaut-robot non-shared space mode, the image capture device and gesture recognition system need to be installed on the console. Select the AAR-2 as the space robot [15,16]. This robot works at the space station and is utilized to assist astronauts to complete some space missions.…”

Section: Experiments Platformmentioning

confidence: 99%

Hand Detection and Location Based on Improved SSD for Space Human-Robot Interaction

Intelligent Robotics and Applications

et al. 2018

Self Cite

In the astronaut-space robot interaction using hand gestures, the detection and location of hands are the premise and basis of vision-based hand gesture recognition and hand tracking. In this paper, the SSD (Single Shot Multibox Detector) which is a kind of Convolutional Neural Network (CNN) model is utilized to detect and locate astronaut's hands for space human-robot interaction (SHRI) based on hand gestures. First of all, in order to meet the need of hand detection and location, an improved SSD model is designed to detect hands when they are shown as small targets in images. Then, a platform for SHRI is built and a set of hand gestures for SHRI are designed. Finally, the proposed SSD model is validated experimentally on a homemade hand gesture database for proving the superiority of this improved SSD model to small target hands detection.