Eating gestures detection by tracking finger motion

Fei, Fan; Gong, Minglun; Lach,

doi:10.1109/wh.2016.7764549

Cited by 7 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of participants ranges from one (i.e., [8,24,28,31,36,44]) to 276 [52] (median: 8 in lab setting, 6 in free-living setting). The total number of participants who successfully participated in the experiments was 1291.…”

Section: Resultsmentioning

confidence: 99%

“…Sixty-one studies (88.4%) used at least one motion sensor on the wrist and five studies (7.2%) reported at least one motion sensor mounted to the lower arm. Four studies [44,58,63,74] used an inertial sensor on a finger in addition to the wrist, while another study [9] only used an accelerometer worn on an index finger. Five studies (7.2%; [23,24,25,26,28]) used motion sensors on the upper arm as well as wrist or lower arm.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

et al. 2019

View full text Add to dashboard Cite

Wearable motion tracking sensors are now widely used to monitor physical activity, and have recently gained more attention in dietary monitoring research. The aim of this review is to synthesise research to date that utilises upper limb motion tracking sensors, either individually or in combination with other technologies (e.g., cameras, microphones), to objectively assess eating behaviour. Eleven electronic databases were searched in January 2019, and 653 distinct records were obtained. Including 10 studies found in backward and forward searches, a total of 69 studies met the inclusion criteria, with 28 published since 2017. Fifty studies were conducted exclusively in laboratory settings, 13 exclusively in free-living settings, and three in both settings. The most commonly used motion sensor was an accelerometer (64) worn on the wrist (60) or lower arm (5), while in most studies (45), accelerometers were used in combination with gyroscopes. Twenty-six studies used commercial-grade smartwatches or fitness bands, 11 used professional grade devices, and 32 used standalone sensor chipsets. The most used machine learning approaches were Support Vector Machine (SVM, n = 21), Random Forest (n = 19), Decision Tree (n = 16), Hidden Markov Model (HMM, n = 10) algorithms, and from 2017 Deep Learning (n = 5). While comparisons of the detection models are not valid due to the use of different datasets, the models that consider the sequential context of data across time, such as HMM and Deep Learning, show promising results for eating activity detection. We discuss opportunities for future research and emerging applications in the context of dietary assessment and monitoring.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

et al. 2019

View full text Add to dashboard Cite

show abstract

“…These studies can be categorized into 3 groups, including food intake detection, food type classification, and food content estimation. Among these groups, food intake detection has been considered as the first phase in food intake monitoring, and studies around it mainly focused on detecting chewing activity (acoustic-based assessment) [7][8][9][10] or hand gestures movement (motion-based assessment) [11][12][13] during eating episodes. The majority of the proposed methods rely on single sensing approaches, for example, using electromyography sensor, accelerometer sensor, or microphone [14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning–Based Multimodal Data Fusion: Case Study in Food Intake Episodes Detection Using Wearable Sensors

Bahador¹,

Ferreira²,

Tamminen³

et al. 2021

JMIR Mhealth Uhealth

View full text Add to dashboard Cite

Background Multimodal wearable technologies have brought forward wide possibilities in human activity recognition, and more specifically personalized monitoring of eating habits. The emerging challenge now is the selection of most discriminative information from high-dimensional data collected from multiple sources. The available fusion algorithms with their complex structure are poorly adopted to the computationally constrained environment which requires integrating information directly at the source. As a result, more simple low-level fusion methods are needed. Objective In the absence of a data combining process, the cost of directly applying high-dimensional raw data to a deep classifier would be computationally expensive with regard to the response time, energy consumption, and memory requirement. Taking this into account, we aimed to develop a data fusion technique in a computationally efficient way to achieve a more comprehensive insight of human activity dynamics in a lower dimension. The major objective was considering statistical dependency of multisensory data and exploring intermodality correlation patterns for different activities. Methods In this technique, the information in time (regardless of the number of sources) is transformed into a 2D space that facilitates classification of eating episodes from others. This is based on a hypothesis that data captured by various sensors are statistically associated with each other and the covariance matrix of all these signals has a unique distribution correlated with each activity which can be encoded on a contour representation. These representations are then used as input of a deep model to learn specific patterns associated with specific activity. Results In order to show the generalizability of the proposed fusion algorithm, 2 different scenarios were taken into account. These scenarios were different in terms of temporal segment size, type of activity, wearable device, subjects, and deep learning architecture. The first scenario used a data set in which a single participant performed a limited number of activities while wearing the Empatica E4 wristband. In the second scenario, a data set related to the activities of daily living was used where 10 different participants wore inertial measurement units while performing a more complex set of activities. The precision metric obtained from leave-one-subject-out cross-validation for the second scenario reached 0.803. The impact of missing data on performance degradation was also evaluated. Conclusions To conclude, the proposed fusion technique provides the possibility of embedding joint variability information over different modalities in just a single 2D representation which results in obtaining a more global view of different aspects of daily human activities at hand, and yet preserving the desired performance level in activity recognition.

show abstract

“…Recently, researchers have used various devices such as wristbands, smartwatches, finger movement sensors, ear-based sensors, glasses or cameras to automatically detect eating action [4,8,9,47,48]. Due to privacy or environmental constraints, the usage of cameras is often not feasible in many scenarios, and thus we focus on the challenges associated with using non-visual sensors that typically include inertial movement units (IMU) and electromyography (EMG).…”

Section: Introductionmentioning

confidence: 99%

“…(iii) Instant feedback is critical to bring about behavioral changes to eating patterns; however, most research works do not perform instant detection of eating action. Most recent works with accelerometer data propose detection after complete data collection [9,47,48], and the works that do propose instant eating action detection use other sensors such as video, data gloves, or finger movement sensors [16,24], which are not easy to utilize.…”

Section: Introductionmentioning

confidence: 99%

A User-adaptive Modeling for Eating Action Identification from Wristband Time Series

Lee

Paudyal

Banerjee

et al. 2019

ACM Trans. Interact. Intell. Syst.

View full text Add to dashboard Cite

Eating activity monitoring using wearable sensors can potentially enable interventions based on eating speed to mitigate the risks of critical healthcare problems such as obesity or diabetes. Eating actions are polycomponential gestures composed of sequential arrangements of three distinct components interspersed with gestures that may be unrelated to eating. This makes it extremely challenging to accurately identify eating actions. The primary reasons for the lack of acceptance of state-of-the-art eating action monitoring techniques include the following: (i) the need to install wearable sensors that are cumbersome to wear or limit the mobility of the user, (ii) the need for manual input from the user, and (iii) poor accuracy in the absence of manual inputs. In this work, we propose a novel methodology, IDEA, that performs accurate eating action identification within eating episodes with an average F1 score of 0.92. This is an improvement of 0.11 for precision and 0.15 for recall for the worst-case users as compared to the state of the art. IDEA uses only a single wristband and provides feedback on eating speed every 2 min without obtaining any manual input from the user. CCS Concepts: • Human-centered computing → Ubiquitous and mobile computing;

show abstract

Eating gestures detection by tracking finger motion

Cited by 7 publications

References 15 publications

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

Deep Learning–Based Multimodal Data Fusion: Case Study in Food Intake Episodes Detection Using Wearable Sensors

A User-adaptive Modeling for Eating Action Identification from Wristband Time Series

Contact Info

Product

Resources

About