A glasses-type wearable device for monitoring the patterns of food intake and facial activity

Chung, Jungman; Chung, Jungmin; Oh, Won-Jun; Yoo, Yongkyu; Lee, Won Gu; Bang, Hyunwoo

doi:10.1038/srep41690

Cited by 46 publications

(53 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GlasSense is a wearable system developed to recognize different facial activities by monitoring the movement of the temple [12]. GlasSense has two load cells embedded in the hinges of a 3D printed eyeglasses to measure the temporalis muscle activity, This signal is used to recognize facial activities such as: chewing, talking, head movement, and winking.…”

Section: Related Workmentioning

confidence: 99%

EarBit

Bedri

Haynes

et al. 2017

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

161

View full text Add to dashboard Cite

Chronic and widespread diseases such as obesity, diabetes, and hypercholesterolemia require patients to monitor their food intake, and food journaling is currently the most common method for doing so. However, food journaling is subject to self-bias and recall errors, and is poorly adhered to by patients. In this paper, we propose an alternative by introducing EarBit, a wearable system that detects eating moments. We evaluate the performance of inertial, optical, and acoustic sensing modalities and focus on inertial sensing, by virtue of its recognition and usability performance. Using data collected in a simulated home setting with minimum restrictions on participants’ behavior, we build our models and evaluate them with an unconstrained outside-the-lab study. For both studies, we obtained video footage as ground truth for participants activities. Using leave-one-user-out validation, EarBit recognized all the eating episodes in the semi-controlled lab study, and achieved an accuracy of 90.1% and an F1-score of 90.9% in detecting chewing instances. In the unconstrained, outside-the-lab evaluation, EarBit obtained an accuracy of 93% and an F1-score of 80.1% in detecting chewing instances. It also accurately recognized all but one recorded eating episodes. These episodes ranged from a 2 minute snack to a 30 minute meal.

show abstract

Section: Related Workmentioning

confidence: 99%

EarBit

Bedri

Haynes

et al. 2017

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

161

View full text Add to dashboard Cite

show abstract

“…Chung et al incorporated a force-sensitive load cell in eyeglasses hinges to monitor temple movement during chewing, head movement, talking, and winking. A classification of these activities yielded an F1 score of 94% [18]. Farooq et al attached a strain sensor at the temporalis muscle area to obtain chewing cycle information [19].…”

Section: Related Workmentioning

confidence: 99%

Retrieval and Timing Performance of Chewing-Based Eating Event Detection in Wearable Sensors

Zhang

Amft

2020

Sensors

View full text Add to dashboard Cite

We present an eating detection algorithm for wearable sensors based on first detecting chewing cycles and subsequently estimating eating phases. We term the corresponding algorithm class as a bottom-up approach. We evaluated the algorithm using electromyographic (EMG) recordings from diet-monitoring eyeglasses in free-living and compared the bottom-up approach against two top-down algorithms. We show that the F1 score was no longer the primary relevant evaluation metric when retrieval rates exceeded approx. 90%. Instead, detection timing errors provided more important insight into detection performance. In 122 hours of free-living EMG data from 10 participants, a total of 44 eating occasions were detected, with a maximum F1 score of 99.2%. Average detection timing errors of the bottom-up algorithm were 2.4 ± 0.4 s and 4.3 ± 0.4 s for the start and end of eating occasions, respectively. Our bottom-up algorithm has the potential to work with different wearable sensors that provide chewing cycle data. We suggest that the research community report timing errors (e.g., using the metrics described in this work).

show abstract

“…Different from other food volume estimation methods [8]- [11] which only use the camera to collect data, MUSEFood uses multiple sensors on smartphone including camera and microphone. This section is divided into two steps: Food Image Sensing and Audio Sensing.…”

Section: B Sensingmentioning

confidence: 99%

MUSEFood: Multi-Sensor-Based Food Volume Estimation on Smartphones

Gao

Tan

et al. 2019

2019 IEEE SmartWorld, Ubiquitous Intelligence &Amp; Computing, Advanced &Amp; Trusted Computing, Scalable Computing &Amp; Commu

View full text Add to dashboard Cite

Researches have shown that diet recording can help people increase awareness of food intake and improve nutrition management, and thereby maintain a healthier life. Recently, researchers have been working on smartphone-based diet recording methods and applications that help users accomplish two tasks: record what they eat and how much they eat. Although the former task has made great progress through adopting image recognition technology, it is still a challenge to estimate the volume of foods accurately and conveniently. In this paper, we propose a novel method, named MUSEFood, for food volume estimation. MUSEFood uses the camera to capture photos of the food, but unlike existing volume measurement methods, MUSE-Food requires neither training images with volume information nor placing a reference object of known size while taking photos. In addition, considering the impact of different containers on the contour shape of foods, MUSEFood uses a multi-task learning framework to improve the accuracy of food segmentation, and uses a differential model applicable for various containers to further reduce the negative impact of container differences on volume estimation accuracy. Furthermore, MUSEFood uses the microphone and the speaker to accurately measure the vertical distance from the camera to the food in a noisy environment, thus scaling the size of food in the image to its actual size. The experiments on real foods indicate that MUSEFood outperforms state-of-the-art approaches, and highly improves the speed of food volume estimation.

show abstract

A glasses-type wearable device for monitoring the patterns of food intake and facial activity

Cited by 46 publications

References 22 publications

EarBit

EarBit

Retrieval and Timing Performance of Chewing-Based Eating Event Detection in Wearable Sensors

MUSEFood: Multi-Sensor-Based Food Volume Estimation on Smartphones

Contact Info

Product

Resources

About