A New Metric for Evaluating Semantic Segmentation: Leveraging Global and Contour Accuracy

Abstract

Semantic segmentation of images is an important problem for mobile robotics and autonomous driving because it offers basic information which can be used for complex reasoning and safe navigation. Different solutions have been proposed for this problem along the last two decades, and a relevant increment on accuracy has been achieved recently with the application of deep neural networks for image segmentation. One of the main issues when comparing different neural networks architectures is how to select an appropriate metric to evaluate their accuracy. Furthermore, commonly employed evaluation metrics can display divergent outcomes, and thus it is not clear how to rank different image segmentation solutions. This paper proposes a new metric which accounts for both global and contour accuracy in a simple formulation to overcome the weaknesses of previous metrics. We show with several examples the suitability of our approach and present a comparative analysis of several commonly used metrics for semantic segmentation together with a statistical analysis of their correlation. Several network segmentation models are used for validation with virtual and real benchmark image sequences, showing that our metric captures information of the most commonly used metrics in a single scalar value.

Automatic Mapping of Center Pivot Irrigation Systems from Satellite Images Using Deep Learning

“…We evaluated the performance of our model using four metrics: accuracy score, precision, recall [31], and f 1 -score [32]. The accuracy score is the ratio of the correctly classified elements over all available elements.…”

Automatic COVID-19 lung infected region segmentation and measurement using CT-scans images

“…The possibility of having a lesion absence in the image can be presented, resulting in the consideration of patients as normal and healthy, therefore; no segmentation is required. Besides, we assessed statistically the segmentation quality of the proposed against GraphCut, Watershed, and MSA, by picking: Accuracy [37] , Sensitivity [38] , F-Measure [38] , Precision [39] , MCC (Mathew Correlation Coefficient) [38] , Dice [40] , Jaccard [40] , and Specificity [38] . By definition, higher values on these indexes imply a better quality of segmentation.…”

Thermal Object Detection in Difficult Weather Conditions Using YOLO

“…Standard accuracy metrics for the tasks in the domain of computer vision are accuracy, precision, recall, and Fmeasure [75,76]. For the object detection task, the mean average precision (mAP) measure is also used to evaluate the performance of the models [77].…”