Approach to Image-Based Segmentation of Complex Surfaces Using Machine Learning Tools During Motion of Mobile Robots

Abstract

In development of formation control systems for modular robotic devices, especially relevant is the problem of analysis and, particularly, segmentation of complex surfaces, down which the robotic device will move. An approach based on fine-tuning of a neural network model HRNet was developed to solve the problem of segmentation of complex surfaces. Model fine-tuning was performed based on a custom dataset, which included 15,000 labeled images. The training dataset included the scenes of the following types: scenes with stairways, scenes with even surfaces, scenes with isolated obstacles and with groups of obstacles. Approbation and functional quality assessment of the developed approach were performed based on the test dataset, which included 3000 images with different levels of scene illumination. According to the results of the testing, the developed approach shows decent quality of segmentation on images with even surfaces (IoU = 90.2%) and with stairways (IoU = 71.3%) as well maintains some resilience to the variations in the scene luminosity levels. After fine-tuning, the averaged performance metrics of this neural network model on images with luminosity levels of 100% and 70% increased by 9.1% and 7.3% at average and resulted in 65.5% and 52%, respectively.