A New Approach to Image-Based Estimation of Food Volume

Abstract

Abstract:A balanced diet is the key to a healthy lifestyle and is crucial for preventing or dealing with many chronic diseases such as diabetes and obesity. Therefore, monitoring diet can be an effective way of improving people's health. However, manual reporting of food intake has been shown to be inaccurate and often impractical. This paper presents a new approach to food intake quantity estimation using image-based modeling. The modeling method consists of three steps: firstly, a short video of the food is taken by the user's smartphone. From such a video, six frames are selected based on the pictures' viewpoints as determined by the smartphone's orientation sensors. Secondly, the user marks one of the frames to seed an interactive segmentation algorithm. Segmentation is based on a Gaussian Mixture Model alongside the graph-cut algorithm. Finally, a customized image-based modeling algorithm generates a point-cloud to model the food. At the same time, a stochastic object-detection method locates a checkerboard used as size/ground reference. The modeling algorithm is optimized such that the use of six input images still results in an acceptable computation cost. In our evaluation procedure, we achieved an average accuracy of 92% on a test set that includes images of different kinds of pasta and bread, with an average processing time of about 23 s.

A Systematic Review of Sensor-Based Methodologies for Food Portion Size Estimation

“…This approach doesn't require any prior information on the food shape. Another approach, similar to SLAM, is the structure from motion algorithm (SFM) [94]. This approach produced accurate results for convex food items using a portable sensor.…”

Human-Mimetic Estimation of Food Volume from a Single-View RGB Image Using an AI System

“…Although improvements have been made, the model-based methods suffer from a common problem that an appropriate 3D model must be selected manually, which affects the data processing speed and is not cost-efficient. In order to solve this problem, stereo vision-based methods [3,[13][14][15][16] have been developed to directly reconstruct the 3D food surface from a set of RGB or RGBD (a type of color image that contains the information of depth (D) at each pixel location) images. For example, Puri et al [14] developed a reconstruction method from multiple images of different views.…”

“…For example, Puri et al [14] developed a reconstruction method from multiple images of different views. Similarly, Hassannejad et al [13] generated a point cloud of food using six continuous frames extracted from a short video. To speed up the process of 3D reconstruction, Dehais et al [3] proposed a two-view 3D reconstruction algorithm for food volume estimation.…”

“…For example, the size of an apple must be determined by comparing it with another object with a known size in the same image. Thus, many types of fiducial markers have been used as the scale reference, such as a checkerboard [13] and a credit card like reference [3]. However, these fiducial markers must be carried by the individual and placed near the food before the individual starts eating, which is an unwelcome procedure usually difficult to implement.…”

“…Unfortunately, much of the 3D information is lost in the imaging process where the food as a 3D object is projected to a 2D plane. For this reason, instead of using a single-view image, many researchers turned to the use of multiple images in different views [3,[13][14][15][16]. However, this approach requires the individual to move either the camera or the food in the imaging process, complicating the research effort and possibly modifying the normal behavior of diet intake.…”