A grasps-generation-and-selection convolutional neural network for a digital twin of intelligent robotic grasping

Hu, Weifei; Wang, Chuxuan; Liu, Feixiang; Peng, Xiang; Peng-wen, Sun; Tan, Jianrong

doi:10.1016/j.rcim.2022.102371

Cited by 21 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Redmon et al [ 27 ] proposed a one‐stage regression approach based on AlexNet [ 28 ] that detected the grasp rectangles directly on the entire image. Morrison et al [ 8 ] proposed a generative grasp CNN (GG‐CNN), and several grasp detection models, such as GR–CNN, [ 9 ] Squeeze‐and Excitation Residual Network (SE‐ResUNet), [ 10 ] and GGS‐CNN [ 29 ] have been developed based on GG–CNN. However, GG‐CNN is highly dependent on the depth information and is susceptible to interference from the environment, which requires higher demands on image collection.…”

Section: Related Workmentioning

confidence: 99%

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Han,

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

In this article, the arbitrary‐oriented object detection problem with application in robotic grasping is addressed. A novel Jensen–Shannon divergence (JSD)– You Only Look Once (YOLO) model is proposed, which enables real‐time grasp detection with high performance. The one‐stage object detection network YOLOv5 is modified with a decoupled head, which solves the angle classification problem and rectangle parameter regression problem separately, such that the YOLOv5 network is applicable for robotic grasping and the detection accuracy is significantly improved. A circular smooth label angle classification method is proposed to tackle the boundary discontinuity problem in angle regression, and the periodicity of the angle prediction is guaranteed. A novel Jensen–Shannon intersection of union is designed to calculate the intersection over union of oriented rectangles, which aims to better measure the discrepancies between the prediction and the ground truth and to avoid the singularity problem when two rectangles are not overlapped. Extensive evaluation on the Cornell and visual manipulation relationship dataset datasets demonstrates the effectiveness of the JSD–YOLO model in general robotic grasp operations, with 99.7% and 95.7% image‐wise split accuracy, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Han,

2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

show abstract

“…Developing efficient AI algorithms for physical robots is a critical challenge due to its excessive time consumption, power supply and component limitations for long-term repetitive motions, lack of appropriate VTBs, and so on. An effective solution to this issue is to develop the algorithms through extensive simulations where DT was observed to play viable roles [105] , [106] . Examples included a DT-aided DRL-based policy transfer from simulation to physical robots [105] , and training robots in their DTs for intelligent robot grasping applying grasps-generation-and-selection convolutional neural network (GGS-CNN) with 96.7% and 93.8% success rates for gripping single items and mixed objects, respectively [106] .…”

Section: Recent Trends In Dt Incorporated Roboticsmentioning

confidence: 99%

“…An effective solution to this issue is to develop the algorithms through extensive simulations where DT was observed to play viable roles [105] , [106] . Examples included a DT-aided DRL-based policy transfer from simulation to physical robots [105] , and training robots in their DTs for intelligent robot grasping applying grasps-generation-and-selection convolutional neural network (GGS-CNN) with 96.7% and 93.8% success rates for gripping single items and mixed objects, respectively [106] . Industrial Cloud and Edge Robotics Cloud and edge robotics - both are relatively new aspects of the industry 4.0 paradigm.…”

Section: Recent Trends In Dt Incorporated Roboticsmentioning

confidence: 99%

“…DT-aided AI implementation had been growing consistently in most domains to achieve autonomy through machine vision [66] , robot self learning [100] , intelligent and autonomous data processing [104] . Especially, methods such as robot policy transfer [105] , [106] and CNN [103] had been observed to be thriving in domains such as human-robot interaction and industrial robotics. Such additions had led to many intelligent solutions to boost production, ensure better safety at HRI work spaces, reduce collisions and thus increasing plat efficiency.…”

Section: Research Scopes In Dt Incorporated Roboticsmentioning

confidence: 99%

See 1 more Smart Citation

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

Mazumder¹,

Sahed²,

Tasneem³

et al. 2023

Heliyon

View full text Add to dashboard Cite

“…Since the digital twin can reflect the dynamic mapping of physical products and their digital counterparts, it is considered the next wave of modeling, simulation, and optimization technologies [2]. Since the concept of the digital twin was introduced, it has been applied to many industrial fields, such as intelligent robots [3,4], AGV carts [5],…”

Section: Introductionmentioning

confidence: 99%

Fusion method for digital twin model of a production line

Liu,

Wang,

Huang

et al. 2024

Int J Adv Manuf Technol

View full text Add to dashboard Cite

Digital twins have attracted more and more attention in the past few years. To put digital twins into practice, a large number of modeling approaches have been proposed, vast amounts of data have been collected, and their accuracy has been improving. However, current research has paid insufficient attention to the multi-scale features of the shop floor, which hinders the effective application of the digital twin shop floor. To address the problem of how to achieve effective multilevel and multi-dimensional fusion of digital twin models with production process data, this paper first proposes a structured data modeling framework for sorting out all the production process data collected in real-time; and then proposes a multi-level fusion framework for supporting the fusion of real-time data and twin models from the unit level to the system level. The method judges the parsed received data streams through the full-factor semanticization framework, and at the same time fuses the parsed data streams with the constructed full-factor twin model from multiple dimensions and layers, forming a twin model fusion method with real-time data streams as the blood and twin model as the skeleton. Finally, the micro-assembly-based production shop environment is selected as a case study to verify the correctness and feasibility of the proposed data grooming framework, data, and model fusion method.

show abstract

A grasps-generation-and-selection convolutional neural network for a digital twin of intelligent robotic grasping

Cited by 21 publications

References 13 publications

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Jensen–Shannon Divergence You Only Look Once: A Real‐Time Robotic Grasp Detection Network

Towards next generation digital twin in robotics: Trends, scopes, challenges, and future

Fusion method for digital twin model of a production line

Contact Info

Product

Resources

About