Identifying Reflected Images From Object Detector in Indoor Environment Utilizing Depth Information

Abstract

We observed that mirror reflection severely degrades person detection performance in an indoor environment, which is an essential task for service robots. To address this problem, we propose a new real-time method to identify reflected virtual images in an indoor environment utilizing 3D depth information. Images reflected by the mirror are similar to real objects, so it is a non-trivial task to differentiate them. Conventional object detectors, which do not deal with this problem, obviously recognize reflected images as real objects. The proposed method compares the geometric relationship between the 3D spatial information of the detected object and its surrounding environment where the object locates. It analyzes the layout of surrounding indoor space utilizing semantic segmentation and plane detection method. With the estimated layout of indoor space, detected object candidates are examined whether they are real or reflected images utilizing 3D depth information. To verify the proposed method, a large indoor dataset was newly acquired and examined in a dedicated Living-lab environment. The performance of the algorithm is verified by comparing conventional detectors with the proposed method in the acquired Living-lab dataset.

Benchmarking visual SLAM methods in mirror environments

“…In recent years, machine learning approaches have been used to detect mirrors in single images, such as MirrorNet [65], progressive mirror detection (PMD) [66], and systems described in Refs. [23,24,67]. Datasets containing ground-truth mirror masks have also been provided in Refs.…”

“…Reflections are ubiquitous in many domestic and industrial settings, and are often utilised by the human visual system to help people understand their surroundings. On the other hand, computer vision systems have struggled to recognise reflective surfaces and correctly understand the environment's geometry-this has only recently begun to be addressed [23,24]. Filtering out reflections means discarding information that has the potential to extend a camera's coverage of an environment or object.…”

A Survey of Visual SLAM Methods

“…These surfaces are highly reflective and can lead to distorted measurements or even the failure in object detection. To address this issue, Park et al proposed a novel solution that uses 3D depth information to identify virtual images reflected in real-time within indoor environments [136]. Their technique involves comparing the spatial information of detected objects with their surrounding environment to determine their geometric relationship.…”

Designing a Lightweight Edge-Guided Convolutional Neural Network for Segmenting Mirrors and Reflective Surfaces

“…Despite the ubiquitous presence of mirrors and reflective surfaces in everyday scenes -from indoor rooms to outdoor buildings -existing computer vision systems have difficulty detecting them due to their lack of a consistent distinguishing appearance and the visual similarity of reflections with their surroundings [Par21]. This results in complications in tasks such as robot navigation [And18] and three-dimensional scene reconstruction [Zha18], where approaches to accommodate the presence of mirrors entail having to augment visual information from cameras with cues from specialized hardware, including ultrasonic sensors and dedicated illumination devices [Tin16].…”