Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

Turan, Mehmet; Örnek, Evin Pınar; Ibrahimli, Nail; Giracoglu, Can; Almalioglu, Yasin; Yanik, Mehmet Fatih; Sitti, Metin

doi:10.1109/iros.2018.8593623

Cited by 42 publications

(26 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many researchers have realized that handcrafted features merely encode partial information in WCE images [29] and that deep learning methods are capable of extracting powerful feature representations that can be used in WCE lesion recognition and depth estimation [6, 7, 15, 18, 30–34].…”

Section: Methodsmentioning

confidence: 99%

Deep Convolutional Neural Network for Ulcer Recognition in Wireless Capsule Endoscopy: Experimental Feasibility and Optimization

Wang

Xing

Zhang

et al. 2019

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

Wireless capsule endoscopy (WCE) has developed rapidly over the last several years and now enables physicians to examine the gastrointestinal tract without surgical operation. However, a large number of images must be analyzed to obtain a diagnosis. Deep convolutional neural networks (CNNs) have demonstrated impressive performance in different computer vision tasks. Thus, in this work, we aim to explore the feasibility of deep learning for ulcer recognition and optimize a CNN-based ulcer recognition architecture for WCE images. By analyzing the ulcer recognition task and characteristics of classic deep learning networks, we propose a HAnet architecture that uses ResNet-34 as the base network and fuses hyper features from the shallow layer with deep features in deeper layers to provide final diagnostic decisions. 1,416 independent WCE videos are collected for this study. The overall test accuracy of our HAnet is 92.05%, and its sensitivity and specificity are 91.64% and 92.42%, respectively. According to our comparisons of F1, F2, and ROC-AUC, the proposed method performs better than several off-the-shelf CNN models, including VGG, DenseNet, and Inception-ResNet-v2, and classical machine learning methods with handcrafted features for WCE image classification. Overall, this study demonstrates that recognizing ulcers in WCE images via the deep CNN method is feasible and could help reduce the tedious image reading work of physicians. Moreover, our HAnet architecture tailored for this problem gives a fine choice for the design of network structure.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep Convolutional Neural Network for Ulcer Recognition in Wireless Capsule Endoscopy: Experimental Feasibility and Optimization

Wang

Xing

Zhang

et al. 2019

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

show abstract

“…Some articles have also researched the movement path of capsule robots [28,29,30,31,32,33]. The visual odometer, SLAM, and machine learning methods were adopted in the above article.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…Above all, neither the absolute localization algorithm based on the sensor coordinate system, nor the relative localization algorithm based on the human coordinate system can provide the appropriate tracking results that can be effectively used for further endoscopy operations. Some researchers have utilized the simultaneous localization and mapping (SLAM) method to obtain position information [28,29,30,31,32,33]. Usually, vision information has been used for tracking by applying the learning methods.…”

Section: Introductionmentioning

confidence: 99%

A Novel Relative Position Estimation Method for Capsule Robot Moving in Gastrointestinal Tract

Wang

Shi

Song

et al. 2019

Sensors

View full text Add to dashboard Cite

Recently, a variety of positioning and tracking methods have been proposed for capsule robots moving in the gastrointestinal (GI) tract to provide real-time unobstructed spatial pose results. However, the current absolute position-based result cannot match the GI structure due to its unstructured environment. To overcome this disadvantage and provide a proper position description method to match the GI tract, we here present a relative position estimation method for tracking the capsule robot, which uses the moving distance of the robot along the GI tract to indicate the position result. The procedure of the proposed method is as follows: firstly, the absolute position results of the capsule robot are obtained with the magnetic tracking method; then, the moving status of the robot along the GI tract is determined according to the moving direction; and finally, the movement trajectory of the capsule robot is fitted with the Bézier curve, where the moving distance can then be evaluated using the integral method. Compared to state-of-the-art capsule tracking methods, the proposed method can directly help to guide medical instruments by providing physicians the insertion distance in patients’ bodies, which cannot be done based on absolute position results. Moreover, as relative distance information was used, no reference tracking objects needed to be mounted onto the human body. The experimental results prove that the proposed method achieves a good distance estimation of the capsule robot moving in the simulation platform.

show abstract

“…In [12], the error is reported for the angles up to 45° and it is in the order of 1°. In [15], the absolute rotational error is about 1° for the rotation angles up to 15° and increases up to 7° for the rotation angles up to 40°. The error for the roll angles more than 40° has not been reported.…”

Section: Discussionmentioning

confidence: 99%

New image‐guided method for localisation of an active capsule endoscope in the stomach

et al. 2019

View full text Add to dashboard Cite

Localisation of an active capsule endoscope inside the stomach has different challenges. One of them is the estimation of the capsule's roll angle. Another challenge is adjusting the distance between the capsule and the stomach to achieve high‐quality imaging in the region of interest. In this study, an optimised image‐guided localisation (O‐Localisation) method is proposed to estimate the roll angle and the scale factor between the consecutive frames. The distance between the capsule and walls of the stomach can be adjusted using the suggested fuzzy adjuster, which is developed based on the estimated scale factors and calibration parameters. This new method is only based on visual information extracted from wireless capsule endoscope video frames. The results show that this method can accurately estimate the rotation angles and scale factors with errors <0.2% for the angles up to 90° and 0.3% for the scales up to 5, respectively. The method is robust to the brightness changes up to 80% with a maximum error of 0.3%. The computational time is about 1 s and can be considered near real‐time for this application. Accordingly, the O‐Localisation method as a real‐time, robust and precise method for capsule localisation can provide a more efficient controllable and steerable capsule endoscopes.

show abstract

Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots

Cited by 42 publications

References 28 publications

Deep Convolutional Neural Network for Ulcer Recognition in Wireless Capsule Endoscopy: Experimental Feasibility and Optimization

Deep Convolutional Neural Network for Ulcer Recognition in Wireless Capsule Endoscopy: Experimental Feasibility and Optimization

A Novel Relative Position Estimation Method for Capsule Robot Moving in Gastrointestinal Tract

New image‐guided method for localisation of an active capsule endoscope in the stomach

Contact Info

Product

Resources

About