Online assessment of computer vision and robotics skills based on a digital twin

Rivera-Calderón, Samantha; Lázaro, Rafael Pérez-San; Vazquez-Hurtado, Carlos

doi:10.1109/educon52537.2022.9766459

Cited by 3 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The robot model was visualized and could be controlled from its DT [100] . More recent practices included DT-integrated machine vision-based assessment of industrial robotic skills [101] , and industrial robot programming using machine learning enhanced point cloud information and RDT [102] . A critical shortcoming of data-driven RDT approach as hinted by [99] was lack of historical data.…”

Section: Recent Trends In Dt Incorporated Roboticsmentioning

confidence: 99%

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

Mazumder¹,

Sahed²,

Tasneem³

et al. 2023

Heliyon

View full text Add to dashboard Cite

Section: Recent Trends In Dt Incorporated Roboticsmentioning

confidence: 99%

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

Mazumder¹,

Sahed²,

Tasneem³

et al. 2023

Heliyon

View full text Add to dashboard Cite

“…According to the literature, researchers have shown that utilizing virtual scenes can be a replacement or comparison source to their real-world counterparts. As addressed by Rivera-Calderón et al in [1], a lack of resources and accessibility to real-world scenes can be an issue in training robots in the digital twin. For instance, in [1], the visualization of a virtual robot that is controlled based on data captured from cameras in the physical environment was introduced to allow for the integration of knowledge and remote practice without the need for a physical robot, thereby increasing equity and inclusivity in assessment.…”

Section: Introductionmentioning

confidence: 99%

“…As addressed by Rivera-Calderón et al in [1], a lack of resources and accessibility to real-world scenes can be an issue in training robots in the digital twin. For instance, in [1], the visualization of a virtual robot that is controlled based on data captured from cameras in the physical environment was introduced to allow for the integration of knowledge and remote practice without the need for a physical robot, thereby increasing equity and inclusivity in assessment. Bansal et al in [2] argued that advanced manufacturers often require dynamic changes on the factory floor to enable manufacturing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

RoboTwin Metaverse Platform for Robotic Random Bin Picking

Tsai,

Hernandez,

You

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Although vision-guided robotic picking systems are commonly used in factory environments, achieving rapid changeover for diverse workpiece types can still be challenging because the manual redefinition of vision software and tedious collection and annotation of datasets consistently hinder the automation process. In this paper, we present a novel approach for rapid workpiece changeover in a vision-guided robotic picking system using the proposed RoboTwin and FOVision systems. The RoboTwin system offers a realistic metaverse scene that enables tuning robot movements and gripper reactions. Additionally, it automatically generates annotated virtual images for each workpiece’s pickable point. These images serve as training datasets for an AI model and are deployed to the FOVision system, a platform that includes vision and edge computing capabilities for the robotic manipulator. The system achieves an instance segmentation mean average precision of 70% and a picking success rate of over 80% in real-world detection scenarios. The proposed approach can accelerate dataset generation by 80 times compared with manual annotation, which helps to reduce simulation-to-real gap errors and enables rapid line changeover within flexible manufacturing systems in factories.

show abstract

“…These systems use images and videos to understand real-world scenes [1,2]. In robotics, computer vision techniques can be used, where the camera acts as a sensor for the robot [3]. The camera collects data from the external environment and applies algorithms to the collected data of a particular object to control the applied system [4].…”

Section: Introductionmentioning

confidence: 99%

Guidance the Wall Painting Robot Based on a Vision System

Jasim¹,

Al-Isawi²,

Hamad³

2022

JESA

View full text Add to dashboard Cite

The spraying operation is one of the most important processes in which industrial robots should be used, the most important of which is spraying for the purpose of painting walls, cars, and devices, in addition to spraying insecticides on plants to get rid of agricultural pests and others. An autonomous spraying robot is intended to alleviate numerous challenges associated with hand spraying. The proposed robot is a wall painting cartesian robot's conceptual design, which includes a paint object with a spray gun and a vision system. The cartesian robot has three links which are X, Y, and Z axes. The spray gun is connected to a screw, which causes the link to move linearly. When the spray gun reaches a particular limit, the camera detects it. The robot needs an appropriate trajectory to prevent collisions with other objects, pass a defined point in spatial coordinates, and accomplish rapid and precise mobility. Through the camera, the robot coordinates mapping, identifies non-sprayable places such as windows or doors and then inspects the spraying effect. The experimental results were applied to four maps (flat map, door map, window map, door and window map, and door and window map), and the corner locations for each map were identified using the vision system. Finally, by comparing the results to the actual distance, the lengths between the corners were computed.

show abstract

Online assessment of computer vision and robotics skills based on a digital twin

Cited by 3 publications

References 18 publications

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

RoboTwin Metaverse Platform for Robotic Random Bin Picking

Guidance the Wall Painting Robot Based on a Vision System

Contact Info

Product

Resources

About