Fully Automatic and Real-Time Microrobot Detection and Tracking based on Ultrasound Imaging using Deep Learning

Botros, Karim; Alkhatib, Mohammad; Folio, David; Ferreira, Antoine

doi:10.1109/icra46639.2022.9812114

Cited by 10 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CNN is the most applied DL method followed by Long Short Term Memory (LSTM), Recurrent Neural Network (RNN) and autoencoder architectures (Figure 5). The CNN model and the variants, with few modifications in the underlying architecture in some cases, yielded better performance in [29], [67]- [70], [72]- [75], [77], [79], [80], [82]- [85], [87]- [92], [94], [97], [98], [100], [101], whereas autoencoders, RNN, LSTM, and Generative Adversarial Network (GAN) formed another notable synergy [39], [71], [76], [78], [81], [86], [93], [95].…”

Section: ) Tool Detectionmentioning

confidence: 99%

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

et al. 2022

View full text Add to dashboard Cite

The recent advancements in the surging field of Deep Learning (DL) have revolutionized every sphere of life, and the healthcare domain is no exception. The enormous success of DL models, particularly with image data, has led to the development of image-guided Robot Assisted Surgery (RAS) systems. By and large, the number of studies concerning image-driven computer assisted surgical systems using DL has increased exponentially. Additionally, the contemporary availability of surgical datasets has also boosted the DL applications in RAS. Inspired by the latest trends and contributions in surgery, this literature survey presents a summarized analysis of recent innovations of DL in image-guided RAS systems. After a thorough review, a sum of 184 articles are selected and grouped into four categories, based on the literature and the relevancy of the task in the articles, comprising 1) Surgical Tools, 2) Surgical Processes, 3) Surgical Surveillance, and 4) Surgical Performance. The survey also discusses publicly available surgical datasets and highlights the basics of the DL models. Furthermore, the legal, ethical, and technological challenges together with the intuitive predictions and recommendations related to the autonomous RAS systems are also presented. The study reveals that Convolutional Neural Network (CNN) is most widely adopted architecture, whereas, JIGSAWS is most employed dataset in RAS. The study suggests fusing kinematic data along with image data, which produces better accuracy and precision, particularly in gesture and trajectory segmentation tasks. Additionally, CNN and Long Short Term Memory (LSTM) networks have shown remarkable performance, however, authors recommend employing these gigantic architectures only when simpler models have failed to produce satisfactory results. The simpler models, despite their limitations, are time and cost effective and yield considerable outcomes even on the smaller datasets.

show abstract

Section: ) Tool Detectionmentioning

confidence: 99%

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Meitin Sitti [ 22 ] exemplified the application of deep learning technology in an endoscopic capsule robot, proposing a localization method grounded in endoscope camera information and multi-sensor fusion. Karim Botros’ team [ 23 ] proposed a chain-like magnetic microsphere robot target detection and tracking system based on ultrasound imaging. This method uses CNN neural networks in deep learning technology to estimate the position of the microrobot in real time.…”

Section: Introductionmentioning

confidence: 99%

Magnetic-Controlled Microrobot: Real-Time Detection and Tracking through Deep Learning Approaches

Li,

Yi,

Zhang

et al. 2024

Micromachines

View full text Add to dashboard Cite

As one of the most significant research topics in robotics, microrobots hold great promise in biomedicine for applications such as targeted diagnosis, targeted drug delivery, and minimally invasive treatment. This paper proposes an enhanced YOLOv5 (You Only Look Once version 5) microrobot detection and tracking system (MDTS), incorporating a visual tracking algorithm to elevate the precision of small-target detection and tracking. The improved YOLOv5 network structure is used to take magnetic bodies with sizes of 3 mm and 1 mm and a magnetic microrobot with a length of 2 mm as the pretraining targets, and the training weight model is used to obtain the position information and motion information of the microrobot in real time. The experimental results show that the accuracy of the improved network model for magnetic bodies with a size of 3 mm is 95.81%, representing an increase of 2.1%; for magnetic bodies with a size of 1 mm, the accuracy is 91.03%, representing an increase of 1.33%; and for microrobots with a length of 2 mm, the accuracy is 91.7%, representing an increase of 1.5%. The combination of the improved YOLOv5 network model and the vision algorithm can effectively realize the real-time detection and tracking of magnetically controlled microrobots. Finally, 2D and 3D detection and tracking experiments relating to microrobots are designed to verify the robustness and effectiveness of the system, which provides strong support for the operation and control of microrobots in an in vivo environment.

show abstract

Evolution of Surgical Robot Systems Enhanced by Artificial Intelligence: A Review

Liu,

Wu,

Sang

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Surgical robot systems (SRS) represent an innovative cross‐disciplinary research field using robotic technology to assist surgeons in operations. Current bottlenecks in SRS, such as the limited ability to process complex information and make surgical decisions, have not been effectively solved. Artificial intelligence (AI) is a valuable technique for simulating and extending human intelligence. AI offers a new direction and impetus for SRS by enhancing performance in areas such as perception, navigation, surgical planning, and control strategies. This review introduces the developmental history of AI‐aided SRS, summarizes the basic SRS architecture, and analyzes how AI can improve SRS performance. Classical cases of AI‐aided SRS, the impact of evidence in clinical settings, and associated ethical and legal considerations are explored. Finally, the challenges in AI‐aided SRS are discussed, including algorithm development, surgical data science, human–robot coordination, and trust building between humans and robots.

show abstract

Fully Automatic and Real-Time Microrobot Detection and Tracking based on Ultrasound Imaging using Deep Learning

Cited by 10 publications

References 21 publications

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

Magnetic-Controlled Microrobot: Real-Time Detection and Tracking through Deep Learning Approaches

Evolution of Surgical Robot Systems Enhanced by Artificial Intelligence: A Review

Contact Info

Product

Resources

About