Supporting Healthy Grocery Shopping via Mobile Augmented Reality

Abstract

Augmented reality (AR) applications have recently become popular on modern smartphones. We explore the effectiveness of this mobile AR technology in the context of grocery shopping, in particular as a means to assist shoppers in making healthier decisions as they decide which grocery products to buy. We construct an AR-assisted mobile grocery-shopping application that makes real-time, customized recommendations of healthy products to users and also highlights products to avoid for various types of health concerns, such as allergies to milk or nut products, low-sodium or low-fat diets, and general caloric intake. We have implemented a prototype of this AR-assisted mobile grocery shopping application and evaluated its effectiveness in grocery store aisles. Our application's evaluation with typical grocery shoppers demonstrates that AR overlay tagging of products reduces the search time to find healthy food items, and that coloring the tags helps to improve the user's ability to quickly and easily identify recommended products, as well as products to avoid. We have evaluated our application's functionality by analyzing the data we collected from 15 in-person actual grocery-shopping subjects and 104 online application survey participants.

Characteristics of Smartphone Applications for Nutrition Improvement in Community Settings: A Scoping Review

“…Field deployment (real-time use of the application by consumers in the intended setting), with or without participant observation, was the most common testing design used. This approach was taken in 3 nutritionimprovement trials (36,48,50) and 13 application-development projects (65,68,70,72,74,76,79,80,85,(88)(89)(90)92). Other common approaches were testing of application components (36,52,75,85) and focus groups (73,85,86).…”

“…The size of the testing sample varied greatly. The most common size was between 11 and 20 testers (9 studies) (48,72,79,85,86,(88)(89)(90)92), followed by #10 testers (4 studies) (73,76,85,90), 21-50 testers (3 studies) (36,74,77), 51-100 testers (2 studies) (70, 80), 101-200 testers (2 studies) (75,92), and least commonly >200 testers (1 study) (65).…”

Design and Development of Smart List: A Mobile App for Creating and Managing Grocery Lists

“…Other apps work as a recommender system that could suggest the grocery stores on sale that offers a reasonable price for the grocery items in the list such as the Smart Shopping List (2). The augmented reality (AR) technology was embedded in a system named AR-Assisted Mobile Grocery Shopping (24) to assist users in locating healthy foods at the grocery store aisles.…”

Impact of Mixed Reality Food Labels on Product Selection

“…Until today, studies on MR-mediated interventions for supporting consumers in selecting healthy food items have so far focused on the general design of such novel interventions, mainly demonstrating early-stage prototypes, or conducting field studies using smartphones rather than headsets. For example, smartphone-mediated MR application have been designed to leverage CV in supporting consumers in identifying vegetables [43], to estimate portion sizes of composed dishes [5,37], to help users navigate around the supermarket and discover healthy food items [1]. Smartphone-based MR applications were found to be easy-to-use [1], to alter consumer behavior [4,19], and to positively improve food choices [19], yet still suffer under similar shortcomings when compared to barcode scanning applications, i.e.…”