FAIM: Vision and Weight Sensing Fusion Framework for Autonomous Inventory Monitoring in Convenience Stores

Abstract

A common pain point for physical retail stores is live inventory monitoring, i.e., knowing how many items of each product are left on the shelves. About 4% of sales are lost due to an average 5-10% out-of-shelf stockout rate, while additional supplies existed in the warehouse. Traditional techniques rely on manual inspection, per-item tagging using RFIDs, or human-in-the-loop systems, such as Amazon Go. These approaches, while effective, either have poor accuracy, long delays between results or are cost prohibitive. In this paper, we present FAIM (Autonomous Inventory Monitoring Framework) for cashier-less stores. To the best of our knowledge, this is the first fully autonomous system that fuses multiple sensing modalities. Utilizing weight difference on a shelf, visual item recognition in customers' hands and prior knowledge of item layout FAIM monitors products picked up or returned without human-in-the-loop. We present results from a real-world setup with 85 items (33 unique products) replicating the layout of a local 7-Eleven store. To evaluate our system we characterize the similarity of the unique products across three physical features (i.e., weight, color, and location). Our results show that the fused approach provides up to 92.6% item identification accuracy, a 2× reduction in error compared to reported self-checkout stations.

Multimodal Fine-Grained Grocery Product Recognition Using Image and Ocr Text

“…Recognition of products is required for many types of automation solutions at grocery stores. This includes automatic checkout systems for the registration of products [11][12][13], the monitoring of availability and misplacement on store shelves [14][15][16], frictionless checkout where a camera system with the inclusion of other sensors in a store registers the pick of products by customers [17,18] and detection of barcode switches and other fraudulent actions at SCOs [8,19]. A figure showing examples of applications can be seen in Figure 1.…”