GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery

Abstract

People spend an enormous amount of time and effort looking for lost objects. To help remind people of the location of lost objects, various computational systems that provide information on their locations have been developed. However, prior systems for assisting people in finding objects require users to register the target objects in advance. This requirement imposes a cumbersome burden on the users, and the system cannot help remind them of unexpectedly lost objects. We propose GO-Finder ("Generic Object Finder"), a registration-free wearable camera based system for assisting people in finding an arbitrary number of objects based on two key features: automatic discovery of hand-held objects and image-based candidate selection. Given a video taken from a wearable camera, Go-Finder automatically detects and groups hand-held objects to form a visual timeline of the objects. Users can retrieve the last appearance of the object by browsing the timeline through a smartphone app. We conducted a user study to investigate how users benefit from using GO-Finder and confirmed improved accuracy and reduced mental load regarding the object search task by providing clear visual cues on object locations. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools.

Exploring the Role of Video Playback Visual Cues in Object Retrieval Tasks

“…The system associates static images of objects with the individuals who might possess them, providing users with an alternative method for locating items. Similarly, Yagi et al [35] proposed GO-Finder, a system that automatically identifies and logs items during the object placement process. When users need to find these items later, the system provides a timeline of last appearance images to aid in object retrieval tasks.…”

“…In this study, we evaluate the performance of three types of visual cues in object retrieval tasks: (i) last frame (LF): static image of object last appearance; (ii) video: the normal speed video playback starting from DA starting point; and (iii) video-LF: video playback starting from DA at normal speed and consistently displaying the last frame after completion of the playback. Numerous studies have confirmed the effectiveness of images of objects' last appearance in object retrieval tasks [33,35]. Inspired by these studies on extracting static images of objects' last appearance to serve as visual cues, we utilized the static image of the object's last appearance in the video playback as a visual cue, which was last frame of video playback (LF).…”

D-Tags Design by Combining Bluetooth Router, IoT, and Mobile Phone to Track Personal Items

“…The personal items tracking system is a large-scale innovation that needs improvements. The paper by [6] described another way to accomplish how a tracking system works with a camera-based hand-held system. Moreover, the researchers could obtain Bluetooth communication through various topics such as object tracking, child monitoring, and location detection [7][8][9].…”

Background Mixup Data Augmentation for Hand and Object-in-Contact Detection

“…Detecting the positions of a person's hands and an objectin-contact (hand-object detection) from an image provides an important clue for understanding how the person interacts with the physical world. This hand-object detection is applicable to recognizing a person's primitive actions, such as "taking" or "pushing", and logging the person's activity of interacting with the environment [1]. Shan et al [2] built a hand-object detector for localizing hands and interacting objects on a large-scale dataset collected in naturalistic house-holding situations, such as in kitchen [3,4,5], DIY [2], and craft work [2,5].…”