mid-DeepLabv3+ : A Novel Approach for Image Semantic Segmentation applied to African Food Dietary Assessment

Abstract

Recent decades have seen the development of vision-based dietary assessment systems (VBDA). These systems generally consist of three main stages: food image analysis, portion estimation and nutrient derivation. The effectiveness of the initial step is highly dependent on the use of accurate segmentation and image recognition models, and the availability of high-quality training datasets. Food image segmentation still faces various challenges, while most existing research has focused mainly on Asian and Western food images. For this reason, this study is based on food images from sub-Saharan Africa, which pose their own problems, such as inter-class similarity and dishes presentation with mixed-class food. This work focuses on the first stage of VBDA, where we introduce two notable contributions. Firstly, we propose mid-DeepLabv3+, an enhanced food image segmentation model based on DeepLabv3+ with a ResNet50 backbone. Our approach involves adding a middle layer in the decoder path and SimAM attention mechanism after each extracted backbone features layers. Secondly, we present CamerFood10, the first food images dataset specifically designed for sub-Saharan African food segmentation. It includes 10 classes of most consumed food items in Cameroon. On our dataset, mid-DeepLabv3+ outperforms existing image segmentation benchmark models with an mIoU (mean Intersection over Union) of 65.20\%, representing more than +10.74\% improvement over the DeepLabv3+ with the same backbone.