Top-down Spatial Attention for Visual Search: Novelty Detection-Tracking Using Spatial Memory with a Mobile Robot

Abstract

Assistive robotics technologies have been growing impact on at-home monitoring services to support daily life. One of the main research fields is to develop an autonomous mobile robot with the tasks detection, tracking, observation and analysis of the subject of interest in the indoor environment. The main challenges in such daily monitoring application, thus in visual search, are that the robot should track the subject successfully in several severe varying conditions. Recent color and depth image based visual search methods can help to handle part of the problems, such as changing illumination, occlusion, and etc. but these methods generally use large amount of training data by checking the whole scene with high redundancy to find the region of interest. Therefore, inspired by the idea that spatial memory can reveal novelty regions for finding the attention points as in Human Visual System (HVS), we proposed a simple and novel algorithm that integrates Kinect and Lidar(Light Detection And Ranging) sensor data to detect and track novelties using the environment map of the robot as a topdown approach without the necessity of large amount of training data. Then, novelty detection and tracking is achieved based on space based saliency map representing the novelty on the scene. Experimental results demonstrated that the proposed visual attention based scene analysis can handle various conditions stated and achieve high accuracy of novelty detection and tracking.

Spatial Visual Attention for Novelty Detection: A Space-based Saliency Model in 3D Using Spatial Memory

“…VA mechanism helps us gaze into specific regions within a scene instead of processing all of the visual information on the visual field by decreasing redundancy and encouraging our attention to shift [1], [2], [3], [4]. Inspired by the VA mechanism, many works [4], [5], [6], [7], [8] have aimed for developing computational models to obtain saliency maps (attention maps).…”

“…In our research [7], [8], we have been developing algorithms to integrate visual attention computational models to at-home biomonitoring mobile robot applications for people in need who have difficulties leaving their home. The mobile robot has to detect, track and recognize the activities of the subject [7], [8], [9], [10] during daily activities in an indoor environment.…”

“…The mobile robot has to detect, track and recognize the activities of the subject [7], [8], [9], [10] during daily activities in an indoor environment. Obviously, the vision tasks of such a system are the most crucial parts that have to be robust to detect and track novel regions within a scene (e.g., 1 Graduate School of Engineering, Chiba University, Chiba 263-8522, Japan yuwill@faculty.chiba-u.jp subjects) to enable the system to perform activity recognition task.…”

“…Range sensor based approaches have a certain advantage over stereo vision for low light or dark environmental conditions. Therefore, in our previous study [8], we demonstrated that the spatial memory of the observer can be used to detect and track novel regions in the scene regardless of the c 2015 Information Processing Society of Japan low-level or high-level features of the objects by using Kinect and Lidar (Light Detection and Ranging) range sensors [8]. The algorithm in Ref.…”

“…The algorithm in Ref. [8] compares local observation from the Kinect sensor with the environment memory of the robot as the global 2D map of the place, which is obtained by a Lidar sensor [8]. However, in the spatial attention model of Ref.…”