Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptation

Shao, Zeman; Fang, Shaobo; Mao, Runyu; He, Jiangpeng; Wright, Janine; Kerr, Deborah A.; Boushey, Carol J.; Zhu, Fengqing

doi:10.1109/mmsp53017.2021.9733557

Cited by 12 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used the same training method as described in [12] where the RGB image serves as the input to the generative model which outputs a generated Energy Density Map.…”

Section: B Energy Density Mapmentioning

confidence: 99%

“…Both use the VGG-16 [34] network as their backbone. Previously, VGG-16 was used as the backbone to extract features from the Energy Density Map [12] and Resnet-50 was used as the RGB feature extractor [35]. Since extracting features from a single channel depth map is easier than extracting features from an RGB image, VGG-16 is sufficient in this case.…”

Section: Food Energy Estimationmentioning

confidence: 99%

“…Image-based methods have achieved impressive results in many applications for dietary assessment such as food recognition, food segmentation and food portion estimation [7]- [11]. In this paper, we focus on food energy estimation which is challenging even for humans including domain experts (e.g., dietitians) to accurately estimate the energy of foods in an image without known physical references [12], [13]. Food energy estimation is closely related to image-based food portion estimation, which can be classified into three categories [7]:…”

Section: Introductionmentioning

confidence: 99%

“…A conditional Generative Adversarial Network (cGAN) [14] is used to learn this mapping from the RGB image to the "Energy Distribution Map." In [12], we further adopted the concept of the "Energy Distribution Map" and combined its features with the features extracted from the RGB image to improve the estimation of food energy. The performance is shown to have improved when using a combination of these features as compared to using the "Energy Distribution Map" alone.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Image Based Food Energy Estimation With Depth Domain Adaptation

Vinod

Shao

Zhu

2022

2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR)

Self Cite

View full text Add to dashboard Cite

Assessment of dietary intake has primarily relied on self-report instruments, which are prone to measurement errors. Dietary assessment methods have increasingly incorporated technological advances particularly mobile, image based approaches to address some of these limitations and further automation. Mobile, image-based methods can reduce user burden and bias by automatically estimating dietary intake from eating occasion images that are captured by mobile devices. In this paper, we propose an "Energy Density Map" which is a pixel-to-pixel mapping from the RGB image to the energy density of the food. We then incorporate the "Energy Density Map" with an associated depth map that is captured by a depth sensor to estimate the food energy. The proposed method is evaluated on the Nutrition5k dataset. Experimental results show improved results compared to baseline methods with an average error of 13.29 kCal and average percentage error of 13.57% between the ground-truth and the estimated energy of the food.

show abstract

“…We used the same training method as described in [12] where the RGB image serves as the input to the generative model which outputs a generated Energy Density Map.…”

Section: B Energy Density Mapmentioning

confidence: 99%

Section: Food Energy Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Image Based Food Energy Estimation With Depth Domain Adaptation

Vinod

Shao

Zhu

2022

2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Classification of food images is typically the first and most fundamental step in automated image-based food analysis [8,10]. Most existing works focus on designing methods to improve the accuracy of food classification using static food image datasets [16][17][18][19][20]23]. However, static datasets such as Food-101 [2] or VireoFood-172 [4] are limited to training fixed classifiers, which may not be suitable for real-life scenarios because each person has their unique food consumption patterns.…”

Section: Introductionmentioning

confidence: 99%

Simulating Personal Food Consumption Patterns using a Modified Markov Chain

Pan,

He,

Peng

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Food image classification serves as the foundation of image-based dietary assessment to predict food categories. Since there are many different food classes in real life, conventional models cannot achieve sufficiently high accuracy. Personalized classifiers aim to largely improve the accuracy of food image classification for each individual. However, a lack of public personal food consumption data proves to be a challenge for training such models. To address this issue, we propose a novel framework to simulate personal food consumption data patterns, leveraging the use of a modified Markov chain model and self-supervised learning. Our method is capable of creating an accurate future data pattern from a limited amount of initial data, and our simulated data patterns can be closely correlated with the initial data pattern. Furthermore, we use Dynamic Time Warping distance and Kullback-Leibler divergence as metrics to evaluate the effectiveness of our method on the public Food-101 dataset. Our experimental results demonstrate promising performance compared with random simulation and the original Markov chain method. CCS CONCEPTS• Computing methodologies → Discrete-event simulation; Markov decision processes; • Applied computing → Health informatics.

show abstract