Astronaut Visual Tracking of Flying Assistant Robot in Space Station Based on Deep Learning and Probabilistic Model

Zhang, Rui; Wang, Zhao-Kui; Zhang, Yulin

doi:10.1155/2018/6357185

Cited by 12 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Astronaut movements at future time steps can be predicted by normalbold x ^ k + t Pre = normalbold A k Pre normalbold x ^ k . Note astronaut’s position normalbold ϕ k " can be measured using stereo vision and deep learning-based person detection and tracking techniques, as reported in findings by Zhang et al 26 In addition, the astronaut’s velocity can be obtained from v ' k = ϕ ' k − normalbold ϕ ^ k − 1 Δ t .…”

Section: The Predictive-guide Frameworkmentioning

confidence: 99%

Toward stable astronaut following of extravehicular activity assistant robots using deep reinforcement learning

Zhang

2022

International Journal of Advanced Robotic Systems

Self Cite

View full text Add to dashboard Cite

The use of mobile robots for assisting astronauts in extravehicular activities could be an effective option for improving mission productivity and crew safety. It is thus critical that these robots follow the astronaut and maintain a stable distance to provide personalized and timely assistance. However, most extraterrestrial bodies exhibit rugged terrain that can impede a robot’s movements. As such, a novel predictive-guide following strategy is proposed to improve the stability of astronaut–robot distance in obstructive environments. This strategy combines a deep reinforcement learning navigator and a Kalman filter-based predictor to generate optimized motion sequences for safely following the astronaut and acquire predictive guidance concerning future astronaut movements. The proposed model achieved a success rate of 95.0% in simulated navigation tasks and adapted well to untrained complex environments and varied robot movement settings. Comparative tests indicated our strategy managed to stabilize the following distance to within ±1.0 m of the reference value in obstructed environments, significantly outperforming other following strategies. The feasibility and advantage of the proposed approach was validated with a physical robotic follower in a Mars-like environment. [Formula: see text]

show abstract

Section: The Predictive-guide Frameworkmentioning

confidence: 99%

Toward stable astronaut following of extravehicular activity assistant robots using deep reinforcement learning

Zhang

2022

International Journal of Advanced Robotic Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [207], an object detection and tracking based on background subtraction, optical flow and CAMShift algorithm is presented to track unusual events successfully in video taken by UAV. A visual tracking algorithm based on deep learning and probabilistic model to form Personal Satellite for tracking the astronauts of the space stations in RGB-D videos, reported in [208].…”

Section: Space Sciencementioning

confidence: 99%

Vision Tracking: A Survey of the State-of-the-Art

et al. 2020

View full text Add to dashboard Cite

Vision tracking is a well-studied framework in vision computing. Developing a robust visual tracking system is challenging because of the sudden change in object motion, cluttered background, partial occlusion and camera motion. In this study, the state-of-the art visual tracking methods are reviewed and different categories are discussed. The overall visual tracking process is divided into four stages-object initialization, appearance modeling, motion estimation, and object localization. Each of these stages is briefly elaborated and related researches are discussed. A rapid growth of visual tracking algorithms is observed in last few decades. A comprehensive review is reported on different performance metrics to evaluate the efficiency of visual tracking algorithms which might help researchers to identify new avenues in this area. Various application areas of the visual tracking are also discussed at the end of the study.

show abstract

“…The key challenge of all the above automated techniques is developing defect detection algorithms that are able to perform with accuracy and repeatability. Several attempts have been made and most of them can be divided into the following two categories: the ones that use more traditional image processing techniques [5,[16][17][18] and the ones that use machine learning [19][20][21][22][23][24]. In the first category, image features such as convexity or signal intensity [5] are used.…”

Section: Introductionmentioning

confidence: 99%

Defects Recognition Algorithm Development from Visual UAV Inspections

Avdelidis

Tsourdos

Lafiosca

et al. 2022

Sensors

View full text Add to dashboard Cite

Aircraft maintenance plays a key role in the safety of air transport. One of its most significant procedures is the visual inspection of the aircraft skin for defects. This is mainly carried out manually and involves a high skilled human walking around the aircraft. It is very time consuming, costly, stressful and the outcome heavily depends on the skills of the inspector. In this paper, we propose a two-step process for automating the defect recognition and classification from visual images. The visual inspection can be carried out with the use of an unmanned aerial vehicle (UAV) carrying an image sensor to fully automate the procedure and eliminate any human error. With our proposed method in the first step, we perform the crucial part of recognizing the defect. If a defect is found, the image is fed to an ensemble of classifiers for identifying the type. The classifiers are a combination of different pretrained convolution neural network (CNN) models, which we retrained to fit our problem. For achieving our goal, we created our own dataset with defect images captured from aircrafts during inspection in TUI’s maintenance hangar. The images were preprocessed and used to train different pretrained CNNs with the use of transfer learning. We performed an initial training of 40 different CNN architectures to choose the ones that best fitted our dataset. Then, we chose the best four for fine tuning and further testing. For the first step of defect recognition, the DenseNet201 CNN architecture performed better, with an overall accuracy of 81.82%. For the second step for the defect classification, an ensemble of different CNN models was used. The results show that even with a very small dataset, we can reach an accuracy of around 82% in the defect recognition and even 100% for the classification of the categories of missing or damaged exterior paint and primer and dents.

show abstract

Astronaut Visual Tracking of Flying Assistant Robot in Space Station Based on Deep Learning and Probabilistic Model

Cited by 12 publications

References 31 publications

Toward stable astronaut following of extravehicular activity assistant robots using deep reinforcement learning

Toward stable astronaut following of extravehicular activity assistant robots using deep reinforcement learning

Vision Tracking: A Survey of the State-of-the-Art

Defects Recognition Algorithm Development from Visual UAV Inspections

Contact Info

Product

Resources

About