Monocular Robust Depth Estimation Vision System for Robotic Tasks Interventions in Metallic Targets

Abstract

Robotic interventions in hazardous scenarios need to pay special attention to safety, as in most cases it is necessary to have an expert operator in the loop. Moreover, the use of a multi-modal Human-Robot Interface allows the user to interact with the robot using manual control in critical steps, as well as semi-autonomous behaviours in more secure scenarios, by using, for example, object tracking and recognition techniques. This paper describes a novel vision system to track and estimate the depth of metallic targets for robotic interventions. The system has been designed for on-hand monocular cameras, focusing on solving lack of visibility and partial occlusions. This solution has been validated during real interventions at the Centre for Nuclear Research (CERN) accelerator facilities, achieving 95% success in autonomous mode and 100% in a supervised manner. The system increases the safety and efficiency of the robotic operations, reducing the cognitive fatigue of the operator during non-critical mission phases. The integration of such an assistance system is especially important when facing complex (or repetitive) tasks, in order to reduce the work load and accumulated stress of the operator, enhancing the performance and safety of the mission.

Detection of Velocity Based on Change in the Apparent Size

“…Vision/Camera-based sensors have been reported in the literature for general and robotic applications [1,[3][4][5]. Vision-based sensing systems are developed for positioning, tracking of external objects [3,6].…”

“…Vision/Camera-based sensors have been reported in the literature for general and robotic applications [1,[3][4][5]. Vision-based sensing systems are developed for positioning, tracking of external objects [3,6]. In vision sensing systems, change in spatial position of the object is determined by change in its coordinates in the image of the camera [4,6,7].…”

“…Velocity is one of the important physical quantities which has to be continuously measured in various mechanical systems such as robotic systems. For example, an end effectors position and velocity have to be continuously monitored for the successful operation of the robot [2,3]. Various sensors are available as internal (Encoders, LVDT (Linear Variable Differential Transformer), Tachometer, Hall effect sensor and etc.)…”

MiniCERNBot Educational Platform: Antimatter Factory Mock-up Missions for Problem-Solving STEM Learning

“…The Mechatronics, Robotics and Operations (MRO) section at CERN, part of the Survey, Mechatronics and Measurements (SMM) group of the engineering department is in charge of designing and developing cutting-edge robotics technology to remotely perform real interventions in the accelerator scientific facilities. The CERNBot and Train Inspection Monorail (TIM) platforms are an example of such systems, which are continuously improved and adapted to the CERN necessities, while offering new scientific improvements to the Telerobotics research community [ 7 , 8 , 9 , 10 , 11 ]. The MRO section is part of different CERN’s educational programmes and hosts periodically students between the ages of 16 and 19, from diverse background and education.…”

Multi-Scale Spatio-Temporal Feature Extraction and Depth Estimation from Sequences by Ordinal Classification

“…Depth estimation [1][2][3][4][5][6][7] is a longstanding and fundamental task in 3D computer vision and enables a wide variety of applications, e.g., autonomous driving [8][9][10], Augmented Reality (AR) and Virtual Reality (VR) [11,12], Simultaneous Localization And Mapping (SLAM) [13][14][15][16][17][18], 2D-3D video conversion [19] and 3D scene understanding [20,21]. Most methods to estimate depth fall into three categories: Monocular Depth Estimation, Stereo Depth Estimation and Depth from Motion (Sequence).…”