DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion

IntroductionNutrition is closely related to body health. A reasonable diet structure not only meets the body’s needs for various nutrients but also effectively prevents many chronic diseases. However, due to the general lack of systematic nutritional knowledge, people often find it difficult to accurately assess the nutritional content of food. In this context, image-based nutritional evaluation technology can provide significant assistance. Therefore, we are dedicated to directly predicting the nutritional content of dishes through images. Currently, most related research focuses on estimating the volume or area of food through image segmentation tasks and then calculating its nutritional content based on the food category. However, this method often lacks real nutritional content labels as a reference, making it difficult to ensure the accuracy of the predictions.MethodsTo address this issue, we combined segmentation and regression tasks and used the Nutrition5k dataset, which contains detailed nutritional content labels but no segmentation labels, for manual segmentation annotation. Based on these annotated data, we developed a nutritional content prediction model that performs segmentation first and regression afterward. Specifically, we first applied the UNet model to segment the food, then used a backbone network to extract features, and enhanced the feature expression capability through the Squeeze-and-Excitation structure. Finally, the extracted features were processed through several fully connected layers to obtain predictions for the weight, calories, fat, carbohydrates, and protein content.Results and discussionOur model achieved an outstanding average percentage mean absolute error (PMAE) of 17.06% for these components. All manually annotated segmentation labels can be found at https://doi.org/10.6084/m9.figshare.26252048.v1.

show abstract

Accurate and Reliable Food Nutrition Estimation Based on Uncertainty-Driven Deep Learning Model

Ahn

2024

Applied Sciences

View full text Add to dashboard Cite

Mobile Near-Infrared Spectroscopy (NIR) devices are increasingly being used to estimate food nutrients, offering substantial benefits to individuals with diabetes and obesity, who are particularly sensitive to food intake. However, most existing solutions prioritize accuracy, often neglecting to ensure reliability. This oversight can endanger individuals sensitive to specific foods, as it may lead to significant errors in nutrient estimation. To address these issues, we propose an accurate and reliable food nutrient prediction model. Our model introduces a loss function designed to minimize prediction errors by leveraging the relationships among food nutrients. Additionally, we developed a method that enables the model to autonomously estimate its own uncertainty based on the loss, reducing the risk to users. Comparative experiments demonstrate that our model achieves superior performance, with an R2 value of 0.98 and an RMSE of 0.40, reflecting a 5–15% improvement over other models. The autonomous result rejection mechanism showing a 40.6% improvement further enhances robustness, particularly in handling uncertain predictions. These findings highlight the potential of our approach for precise and trustworthy nutritional assessments in real-world applications.

show abstract

DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion

Cited by 3 publications

References 39 publications

Food Portion Estimation via 3D Object Scaling

Food Portion Estimation via 3D Object Scaling

Visual nutrition analysis: leveraging segmentation and regression for food nutrient estimation

Accurate and Reliable Food Nutrition Estimation Based on Uncertainty-Driven Deep Learning Model

Contact Info

Product

Resources

About