We developed a magnetic-assisted capsule colonoscope system with integration of computer vision-based object detection and an alignment control scheme. Two convolutional neural network models A and B for lumen identification were trained on an endoscopic dataset of 9080 images. In the lumen alignment experiment, models C and D used a simulated dataset of 8414 images. The models were evaluated using validation indexes for recall (R), precision (P), mean average precision (mAP), and F1 score. Predictive performance was evaluated with the area under the P-R curve. Adjustments of pitch and yaw angles and alignment control time were analyzed in the alignment experiment. Model D had the best predictive performance. Its R, P, mAP, and F1 score were 0.964, 0.961, 0.961, and 0.963, respectively, when the area of overlap/area of union was at 0.3. In the lumen alignment experiment, the mean degrees of adjustment for yaw and pitch in 160 trials were 21.70° and 13.78°, respectively. Mean alignment control time was 0.902 s. Finally, we compared the cecal intubation time between semi-automated and manual navigation in 20 trials. The average cecal intubation time of manual navigation and semi-automated navigation were 9 min 28.41 s and 7 min 23.61 s, respectively. The automatic lumen detection model, which was trained using a deep learning algorithm, demonstrated high performance in each validation index.

This paper presents an autonomous navigation system for cost-effective magnetic-assisted colonoscopy, employing force-based sensors, an actuator, a proportional–integrator controller and a real-time heuristic searching method. The force sensing system uses load cells installed between the robotic arm and external permanent magnets to derive attractive force data as the basis for real-time surgical safety monitoring and tracking information to navigate the disposable magnetic colonoscope. The average tracking accuracy on magnetic field navigator (MFN) platform in x-axis and y-axis are 1.14 ± 0.59 mm and 1.61 ± 0.45 mm, respectively, presented in mean error ± standard deviation. The average detectable radius of the tracking system is 15 cm. Three simulations of path planning algorithms are presented and the learning real-time A* (LRTA*) algorithm with our proposed directional heuristic evaluation design has the best performance. It takes 75 steps to complete the traveling in unknown synthetic colon map. By integrating the force-based sensing technology and LRTA* path planning algorithm, the average time required to complete autonomous navigation of a highly realistic colonoscopy training model on the MFN platform is 15 min 38 s and the intubation rate is 83.33%. All autonomous navigation experiments are completed without intervention by the operator.