Automatic visual detection and verification of exterior aircraft elements

Leiva, Javier Ramirez; Villemot, Tanguy; Dangoumeau, Guillaume; Bauda, Marie-Anne; Larnier, Stanislas

doi:10.1109/ecmsm.2017.7945885

Cited by 14 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of cameras and vision-based algorithms for inspecting automated processes has been investigated extensively in the literature. These include the use of various 3D object recognition algorithms [ 12 , 13 , 14 ], Convolutional Neural Networks for corrosion detection of regions [ 15 , 16 ] and the use of knowledge-based vision algorithms to identify regions of interest for closer inspection [ 17 , 18 ]. In a similar way as [ 17 ], we make use of the knowledge of the blade layer structure to develop a transition-based algorithm (discussed in Section 2 ) that enables us to assess and quantify the material state on the blade.…”

Section: Introductionmentioning

confidence: 99%

Applying a 6 DoF Robotic Arm and Digital Twin to Automate Fan-Blade Reconditioning for Aerospace Maintenance, Repair, and Overhaul

Oyekan

Farnsworth

Hutabarat

et al. 2020

Sensors

View full text Add to dashboard Cite

The UK is home to several major air commercial and transport hubs. As a result, there is a high demand for Maintenance, Repair, and Overhaul (MRO) services to ensure that fleets of aircraft are in airworthy conditions. MRO services currently involve heavy manual labor. This creates bottlenecks, low repeatability, and low productivity. Presented in this paper is an investigation to create an automation cell for the fan-blade reconditioning component of MRO. The design and prototype of the automation cell is presented. Furthermore, a digital twin of the grinding process is developed and used as a tool to explore the required grinding force parameters needed to effectively remove surface material. An integration of a 6-DoF industrial robot with an end-effector grinder and a computer vision system was undertaken. The computer vision system was used for the digitization of the fan-blade surface as well as tracking and guidance of material removal. Our findings reveal that our proposed system can perform material removal, track the state of the fan blade during the reconditioning process and do so within a closed-loop automated robotic work cell.

show abstract

Section: Introductionmentioning

confidence: 99%

Applying a 6 DoF Robotic Arm and Digital Twin to Automate Fan-Blade Reconditioning for Aerospace Maintenance, Repair, and Overhaul

Oyekan

Farnsworth

Hutabarat

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…[3][4][5][6] In parallel efforts, significant focus has been on exploring innovative solutions for automating visual inspection via autonomous drones, mobile robots, mounted cameras, and so on. [7][8][9][10] While human factors have been identified to lie at the heart of most of the limitations of visual inspection, [11][12][13] human beings possess distinct capabilities like flexibility, imagination, creativity, and so on, that prove indispensable especially during the inspection of safety-critical components. As such, the key to alleviating the major limitations of visual inspection lies in supplementing human capabilities in a systematic manner (through an appropriate level of automation).…”

Section: Introductionmentioning

confidence: 99%

“…3–6 In parallel efforts, significant focus has been on exploring innovative solutions for automating visual inspection via autonomous drones, mobile robots, mounted cameras, and so on. 7–10…”

Section: Introductionmentioning

confidence: 99%

Augmented reality for enhanced visual inspection through knowledge-based deep learning

Wang

Zargar

Fang

2020

Structural Health Monitoring

View full text Add to dashboard Cite

A two-stage knowledge-based deep learning algorithm is presented for enabling automated damage detection in real-time using the augmented reality smart glasses. The first stage of the algorithm entails the identification of damage prone zones within the region of interest. This requires domain knowledge about the damage as well as the structure being inspected. In the second stage, automated damage detection is performed independently within each of the identified zones starting with the one that is the most damage prone. For real-time visual inspection enhancement using the augmented reality smart glasses, this two-stage approach not only ensures computational feasibility and efficiency but also significantly improves the probability of detection when dealing with structures with complex geometric features. A pilot study is conducted using hands-free Epson BT-300 smart glasses during which two distinct tasks are performed: First, using a single deep learning model deployed on the augmented reality smart glasses, automatic detection and classification of corrosion/fatigue, which is the most common cause of failure in high-strength materials, is performed. Then, in order to highlight the efficacy of the proposed two-stage approach, the more challenging task of defect detection in a multi-joint bolted region is addressed. The pilot study is conducted without any artificial control of external conditions like acquisition angles, lighting, and so on. While automating the visual inspection process is not a new concept for large-scale structures, in most cases, assessment of the collected data is performed offline. The algorithms/techniques used therein cannot be implemented directly on computationally limited devices such as the hands-free augmented reality glasses which could then be used by inspectors in the field for real-time assistance. The proposed approach serves to overcome this bottleneck.

show abstract

“…Although research has been carried out on automated aerial inspection and maintenance on aircraft parts by using UAVs, no studies have been reported so far in the literature. In fact, the majority of the published papers, in this field, report the use of autonomous ground vehicles [16,17] or robotic arms for defects detection [18]. However, such autonomous systems might not be suitable for all the applications, because ground vehicles might not inspect high facilities, while robotic arms might require expensive equipment and appropriate infrastructure.…”

Section: Introductionmentioning

confidence: 99%

Inspection of Aircraft Wing Panels Using Unmanned Aerial Vehicles

Tzitzilonis

Malandrakis

Fragonara³

et al. 2019

Sensors

View full text Add to dashboard Cite

In large civil aircraft manufacturing, a time-consuming post-production process is the non-destructive inspection of wing panels. This work aims to address this challenge and improve the defects’ detection by performing automated aerial inspection using a small off-the-shelf multirotor. The UAV is equipped with a wide field-of-view camera and an ultraviolet torch for implementing non-invasive imaging inspection. In particular, the UAV is programmed to perform the complete mission and stream video, in real-time, to the ground control station where the defects’ detection algorithm is executed. The proposed platform was mathematically modelled in MATLAB/SIMULINK in order to assess the behaviour of the system using a path following method during the aircraft wing inspection. In addition, two defect detection algorithms were implemented and tested on a dataset containing images obtained during inspection at Airbus facilities. The results show that for the current dataset the proposed methods can identify all the images containing defects.

show abstract

Automatic visual detection and verification of exterior aircraft elements

Cited by 14 publications

References 8 publications

Applying a 6 DoF Robotic Arm and Digital Twin to Automate Fan-Blade Reconditioning for Aerospace Maintenance, Repair, and Overhaul

Applying a 6 DoF Robotic Arm and Digital Twin to Automate Fan-Blade Reconditioning for Aerospace Maintenance, Repair, and Overhaul

Augmented reality for enhanced visual inspection through knowledge-based deep learning

Inspection of Aircraft Wing Panels Using Unmanned Aerial Vehicles

Contact Info

Product

Resources

About