An End-to-End Deep Learning System for Recommending Healthy Recipes Based on Food Images

Abstract

Healthy food leads to healthy living and it is a major issue in our days. Nutri-Score is a nutrition label that can be calculated from the nutritional values of a food and helps evaluating the healthiness of it. Nevertheless, we don't always have the nutritional values of the food, so it is not always easy identifying this label. In the same way, it is not easy finding the healthier option to a favorite food. In this paper an end-to end deep learning system is proposed to identify the Nutri-Score label and recommend similar but healthier recipes based on food images. A new dataset of images is extracted from the Recipe 1M and labeled with the Nutri-Score value calculated for each image. Pretrained models Resnet50, Resnet101, EfficientNetB2 and DensNet121 are tuned based on this dataset. The embeddings from the last convolutional layer of the input image are used to find its most similar neighbor based on KNN algorithm. The proposed system suggests recipes with the lowest Nutri-Score similar to the inputted image. Implementations show that the Resnet50 provides highest prediction accuracy.