A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry

Abstract

In the context of dietary management, accurate monitoring of eating habits is receiving increased attention. Wearable sensors, combined with the connectivity and processing of modern smartphones, can be used to robustly extract objective and real-time measurements of human behavior. In particular, for the task of chewing detection, several approaches based on an in-ear microphone can be found in the literature, while other types of sensors have also been reported, such as strain sensors. In this paper, performed in the context of the SPLENDID project, we propose to combine an in-ear microphone with a photoplethysmography (PPG) sensor placed in the ear concha, in a new high accuracy and low sampling rate prototype chewing detection system. We propose a pipeline that initially processes each sensor signal separately, and then fuses both to perform the final detection. Features are extracted from each modality, and support vector machine (SVM) classifiers are used separately to perform snacking detection. Finally, we combine the SVM scores from both signals in a late-fusion scheme, which leads to increased eating detection accuracy. We evaluate the proposed eating monitoring system on a challenging, semifree living dataset of 14 subjects, which includes more than 60 h of audio and PPG signal recordings. Results show that fusing the audio and PPG signals significantly improves the effectiveness of eating event detection, achieving accuracy up to 0.938 and class-weighted accuracy up to 0.892.

The SPLENDID Eating Detection Sensor: Development and Feasibility Study

“…An air microphone placed at the beginning of the ear canal that measures sounds produced by chewing [ 19 - 21 ].…”

“…A photoplethysmogram (PPG) sensor placed on the ear that measures the blood volume in the tissue of the ear, which is affected by chewing activity [ 20 , 21 , 24 ]. This technique has never before been used for this application.…”

“…To make the new version of the eating detection sensor wearable, another device the “datalogger” was added to it [ 21 ]. It houses a data acquisition system, a battery, and an accelerometer.…”

Retrieval and Timing Performance of Chewing-Based Eating Event Detection in Wearable Sensors

“…Merck et al realised a multi-device monitoring system involving in-ear audio, head motion, and wrist motion sensors, which could recognise eating with 92% precision and 89% recall [9]. Papapanagiotou et al proposed an ear-worn eating monitoring system based on PPG, audio and accelerometer, achieving an accuracy up to 93.8% and class-weighted accuracy up to 89.2% in eating detection [10]. Bedri et al used an ear-worn system for chewing instance detection.…”

Modeling Wrist Micromovements to Measure In-Meal Eating Behavior From Inertial Sensor Data

“…Methods that approach food intake monitoring via the chewing mechanism are capable of detecting chewing episodes [7]- [11] and estimating the weight of the bite [12]. Such methods typically make use of acoustic sensors such as inear microphones [7]- [9], strain sensors [10], or a combination of sensors such as accelerometry and audio [11]. Approaches based on swallowing detection measure food intake by capturing the movements of the muscles located in the pharynx and esophagus.…”