A stereoscopic eye-in-hand vision system for remote handling in ITER

Niu, Longchuan; Aha, Liisa; Mattila, Jouni; Gotchev, Atanas; Ruiz, Emilio Olías

doi:10.1016/j.fusengdes.2019.03.036

Cited by 11 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stereoscopic vision and point cloud are two methods of providing stereo-depth information and have been applied in various telerobotic systems for remote vision [50][51][52]. Binocular visual feedback for robotic teleoperation has been widely studied on unmanned aerial vehicles and mobile robots [53,54]. Leveraging depth perception in remote robot teleoperation tasks for general object manipulation is not well studied on mobile manipulators.…”

Section: Intuitive and Natural Teleoperationmentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

There is an increasingly urgent need for humans to interactively control robotic systems to perform increasingly precise remote operations, concomitant with the rapid development of space exploration, deep-sea discovery, nuclear rehabilitation and management, and robotic-assisted medical devices. The potential high value of medical telerobotic applications was also evident during the recent coronavirus pandemic and will grow in future. Robotic teleoperation satisfies the demands of the scenarios in which human access carries measurable risk, but human intelligence is required. An effective teleoperation system not only enables intuitive human-robot interaction (HRI) but ensures the robot can also be operated in a way that allows the operator to experience the “feel” of the robot working on the remote side, gaining a “sense of presence”. Extended reality (XR) technology integrates real-world information with computer-generated graphics and has the potential to enhance the effectiveness and performance of HRI by providing depth perception and enabling judgment and decision making while operating the robot in a dynamic environment. This review examines novel approaches to the development and evaluation of an XR-enhanced telerobotic platform for intuitive remote teleoperation applications in dangerous and difficult working conditions. It presents a strong review of XR-enhanced telerobotics for remote robotic applications; a particular focus of the review includes the use of integrated 2D/3D mixed reality with haptic interfaces to perform intuitive remote operations to remove humans from dangerous conditions. This review also covers primary studies proposing Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) solutions where humans can better control or interact with real robotic platforms using these devices and systems to extend the user’s reality and provide a more intuitive interface. The objective of this article is to present recent, relevant, common, and accessible frameworks implemented in research articles published on XR-enhanced telerobotics for industrial applications. Finally, we present and classify the application context of the reviewed articles in two groups: mixed reality–enhanced robotic telemanipulation and mixed reality–enhanced robotic tele-welding. The review thus addresses all elements in the state of the art for these systems and ends with recommended research areas and targets. The application range of these systems and the resulting recommendations is readily extensible to other application areas, such as remote robotic surgery in telemedicine, where surgeons are scarce and need is high, and other potentially high-risk/high-need scenarios.

show abstract

Section: Intuitive and Natural Teleoperationmentioning

confidence: 99%

Integrating Virtual, Mixed, and Augmented Reality into Remote Robotic Applications: A Brief Review of Extended Reality-Enhanced Robotic Systems for Intuitive Telemanipulation and Telemanufacturing Tasks in Hazardous Conditions

Su,

Chen,

Zhou

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Simulating the visual attention mechanism in machines is termed as visual saliency prediction. Automatic saliency prediction has applications in several research and practical fields including computer vision, robotics, healthcare, and multimedia [2]- [5].…”

Section: Introductionmentioning

confidence: 99%

Saliency Prediction in Uncategorized Videos Based on Audio-Visual Correlation

Qamar

Muneeb

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Substantial research has been done in saliency modeling to make intelligent machines that can perceive and interpret their surroundings and focus only on the salient regions in a visual scene. But existing spatio-temporal saliency models either treat videos as merely image sequences excluding any audio information or are unable to cope with inherently varying content. Based on the hypothesis that an audiovisual saliency model will perform better than traditional spatio-temporal saliency models, this work aims to provide a generic preliminary audio/video saliency model. This is achieved by augmenting visual saliency map with an audio saliency map computed by synchronizing low-level audio and visual features. The proposed model was evaluated using different criteria against eye fixations data for a publicly available video dataset DIEM. The evaluation results show that the model outperforms two state-of-the-art visual spatio-temporal saliency models supporting our hypothesis that an audiovisual model performs better in comparison to a visual model for natural uncategorized videos.Interestingly the effect of audio stimuli is proved to be relevant to human eye movements, for instance in [16] the authors find eye movements to be spatially biased towards the source of audio by performing an eye tracking experiment on images with spatially localized sound sources in three conditions: auditory (A), visual (V) and audio-visual (AV). Moreover, another study [17] analyzed the effects of different type of sounds on human gaze involving an experiment with 36 participants and thirteen sound classes under audio-visual and visual conditions. The sound classes are further clustered into on-screen with one sound source,

show abstract

“…To address the challenges described herein, the 3D node system was proposed in earlier works [6,7,8]. This solution usesd the information provided by a stereo camera pair to estimate the geometry of the knuckle through stereo matching.…”

Section: Introductionmentioning

confidence: 99%