Improving the Recognition of Eating Gestures Using Intergesture Sequential Dependencies

Abstract

This paper considers the problem of recognizing eating gestures by tracking wrist motion. Eating gestures are activities commonly undertaken during the consumption of a meal, such as sipping a drink of liquid or using utensils to cut food. Each of these gestures causes a pattern of wrist motion that can be tracked to automatically identify the activity. Previous works have studied this problem at the level of a single gesture. In this paper, we demonstrate that individual gestures have sequential dependence. To study this, three types of classifiers were built: 1) a K-nearest neighbor classifier which uses no sequential context, 2) a hidden Markov model (HMM) which captures the sequential context of subgesture motions, and 3) HMMs that model intergesture sequential dependencies. We built first-order to sixth-order HMMs to evaluate the usefulness of increasing amounts of sequential dependence to aid recognition. On a dataset of 25 meals, we found that the baseline accuracies for the KNN and the subgesture HMM classifiers were 75.8% and 84.3%, respectively. Using HMMs that model intergesture sequential dependencies, we were able to increase accuracy to up to 96.5%. These results demonstrate that sequential dependencies exist between eating gestures and that they can be exploited to improve recognition accuracy.

Validation of a Deep Learning System for the Full Automation of Bite and Meal Duration Analysis of Experimental Meal Videos

“…On that level, RABiD outperforms all previous comparable efforts of video-based and wrist-worn accelerometer meal analyses. Earlier video-based methods employed spectral segmentation, random forest classification [37], and hidden Markov models in order to quantify dependencies between hand gestures and bite instances [38], with satisfactory agreement (0.60 < κ < 0.80) on bite detection results. More recent bite detection systems, using wrist-worn sensors, deploy deep learning methodologies [35,39] in order to propose more accurate and robust solutions and produce promising results.…”

Modeling Wrist Micromovements to Measure In-Meal Eating Behavior From Inertial Sensor Data

“…The work of Ramos-Garcia et al [28] presents a method that uses the information from wrist-mounted triaxial accelerometers and gyroscopes with the purpose of recognizing certain coarse in-meal gestures including: a) rest, b) utensiling, c) drink, d) bite and e) other. The authors propose a gesture-togesture Hidden Markov Model (HMM) approach for capturing the temporal dependencies between gestures.…”

Recognizing Eating from Body-Worn Sensors

“…One study of two participants achieved 87% accuracy for identifying eating gestures in the lab [1]. More recently, Ramos-Garcia et al [22] collected wrist motion data for 25 participants at an instrumented table in a university cafeteria to allow a more realistic environment with wider food choices. However, no data was collected outside of meals (where other wrist motions may be confounded with eating) and using a single environment limits understanding of the method's generalizability.…”