FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input

Luo, Yuhua; Kim, Young‐Ho; Lee, Bongshin; Hassan, Naeemul; Choe, Eun Kyoung

doi:10.1145/3461778.3462074

Cited by 21 publications

(15 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…However, these were not interactive prototypes. Previous studies have shown that speech input can be visualized for self-tracking [32,33]. We have yet to see the range of laughter sound visualizations and their potential applications, which led to the current study.…”

Section: Background and Related Workmentioning

confidence: 99%

Exploring Laughter Sound Visualizations for Self Reflection

Yang

Ryokai

2022

Designing Interactive Systems Conference

View full text Add to dashboard Cite

We present an exploratory design study on visualizing laughter sounds for personal reflection. We experimented with a variety of graphic design elements to visualize temporal, spatial, and social aspects of laughter sounds. In order to experience their own laughter being visualized, our participants collected audio recordings of everyday conversations with their loved ones. We extracted laughter from the participants' audio files using a machine learning algorithm, then visualized selected laughter in five different types of visual representations, and shared the result with each participant. Through the journey of collecting, seeing, listening to, and interacting with their personal laughter visualizations, participants explored what laughter means for them in different contexts. The study reveals that interactive laughter visualizations have the potential to evoke memories, support emotional expressions, and promote relationships.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Exploring Laughter Sound Visualizations for Self Reflection

Yang

Ryokai

2022

Designing Interactive Systems Conference

View full text Add to dashboard Cite

show abstract

“…Speech input requires little to no screen space and researchers found that speech commands can be easier to perform than using graphical widgets on mobile devices (e.g., [60,113]). Recent work has shown promise for speech input for in-situ data collection on digital devices (e.g., exercise logging on a smart speaker [72], food journaling on a smartphone [71]). For example, Luo and colleagues deployed a speech-based mobile food journal and found that participants provided detailed and elaborate information on their food decisions and meal contexts, with a low perceived capture burden [71].…”

Section: Collecting In-situ Behavioral Datamentioning

confidence: 99%

“…Recent work has shown promise for speech input for in-situ data collection on digital devices (e.g., exercise logging on a smart speaker [72], food journaling on a smartphone [71]). For example, Luo and colleagues deployed a speech-based mobile food journal and found that participants provided detailed and elaborate information on their food decisions and meal contexts, with a low perceived capture burden [71]. Using speech input on a smartwatch poses great potential for lowering the data capture burden while enhancing response rate in EMA studies.…”

Section: Collecting In-situ Behavioral Datamentioning

confidence: 99%

MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech

Kim

Chou

Lee

et al. 2022

CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…While manual food logging using pencil-and-paper based techniques could be useful for self-reflection, mobile food diaries allow logging finegrained details (weight and size of dishes, variety of dishes) about eating episodes in a more systematic way by searching food types, varieties, and sizes in a database and logging them [4]. Some recent mobile food diaries have also looked into easing the process of logging using speech and photos [39]. Going a step further, some studies looked into estimating the calorie intake [40] and also calorie deficit by combining information from food diaries and passively detected physical activities [41].…”

Section: B Mobile Food Diariesmentioning

confidence: 99%

Sensing Eating Events in Context: A Smartphone-Only Approach

et al. 2022

View full text Add to dashboard Cite

While the task of detecting eating events has been examined in prior work using a variety of wearable devices, the use of the smartphone as a standalone device to infer eating events remains as an open issue. In this paper, we propose a framework that infers eating vs. non-eating events from passive smartphone sensing, and evaluated on a dataset of 58 college students. First, we show that time of the day and features from modalities such as screen usage, accelerometer, app usage, and location are indicative of eating and non-eating. Then, we show that eating events can be inferred with an AUROC (area under the receiver operating characteristics curve) of 0.65 (F1-score of 0.75) using a subject-independent machine learning model, which can be further improved up to 0.81 (F1-score of 0.85 for subject-dependent and 0.81 for hybrid models) using personalization techniques. We also show that users have different behavioral and contextual routines around eating episodes, that require specific feature groups in training fully personalized models. These findings are of potential value for future mobile food diary apps that are context-aware by: (i) enabling scalable sensing-based eating studies using only smartphones; (ii) detecting under-reported eating events, thus increasing data quality in self report-based studies; (iii) providing functionality to track food consumption and generate reminders for on-time collection of food diaries; and (iv) supporting mobile interventions towards healthy eating practices.

show abstract

FoodScrap: Promoting Rich Data Capture and Reflective Food Journaling Through Speech Input

Cited by 21 publications

References 52 publications

Exploring Laughter Sound Visualizations for Self Reflection

Exploring Laughter Sound Visualizations for Self Reflection

MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech

Sensing Eating Events in Context: A Smartphone-Only Approach

Contact Info

Product

Resources

About