Deep Room Recognition Using Inaudible Echos

Abstract

Recent years have seen the increasing need of location awareness by mobile applications. This paper presents a room-level indoor localization approach based on the measured room's echos in response to a two-millisecond single-tone inaudible chirp emitted by a smartphone's loudspeaker. Different from other acoustics-based room recognition systems that record full-spectrum audio for up to ten seconds, our approach records audio in a narrow inaudible band for 0.1 seconds only to preserve the user's privacy. However, the short-time and narrowband audio signal carries limited information about the room's characteristics, presenting challenges to accurate room recognition. This paper applies deep learning to effectively capture the subtle fingerprints in the rooms' acoustic responses. Our extensive experiments show that a two-layer convolutional neural network fed with the spectrogram of the inaudible echos achieve the best performance, compared with alternative designs using other raw data formats and deep models. Based on this result, we design a RoomRecognize cloud service and its mobile client library that enable the mobile application developers to readily implement the room recognition functionality without resorting to any existing infrastructures and add-on hardware. Extensive evaluation shows that RoomRecognize achieves 99.7%, 97.7%, 99%, and 89% accuracy in differentiating 22 and 50 residential/office rooms, 19 spots in a quiet museum, and 15 spots in a crowded museum, respectively. Compared with the state-of-the-art approaches based on support vector machine, RoomRecognize significantly improves the Pareto frontier of recognition accuracy versus robustness against interfering sounds (e.g., ambient music).

PhyAug

“…Smart voice-based appliances can exploit RIR as the first principle for effective adaptations to the deployment environments. Acoustic-based indoor localization with deep learning [22] can exploit RIR to reduce target-domain training data sampling complexity. ■ Computational fluid dynamics (CFD) describes the thermal processes in indoor spaces (e.g., data centers).…”

Hivemind

“…Thus, the AoI of this family of actuators are intrinsically confined by the spatial division such as a room. So, the AoI could be estimated by adopting room-level localization techniques [10,12,22,59,115,118,130] or proximity detection (e.g., BLE beacons). Even for a very large place with uncertain borders (e.g., airport terminals, hotel lobbies), the AoI of an actuator could be statically determined by building-wide HVAC simulation tools [31].…”

Indoor Smartphone SLAM with Learned Echoic Location Features

“…Hence, identifying new modalities from smartphones' common built-in hardware to enrich the location sensing toolbox has been an interest of research. Exploiting smartphone's built-in audio hardware for active location sensing receives increasing research [34,43,45,48,52,63]. The active sensing uses smartphone's loudspeaker to emit excitation sounds in the target indoor space and microphone to capture the echoes from the surroundings.…”

“…Thus, the existing studies following the analytic approach often make simplifying assumptions that the major reflectors are at most two nearby walls [34,43,63]. The second category, namely, fingerprint approach [45,48,52], uses the echoes captured by the smartphone at a certain location as the fingerprint of the location and then applies supervised learning to build a location recognition model. However, the blanket process of collecting labeled fingerprints at spatially fine-grained locations to form the training dataset imposes a high overhead.…”

“…However, the blanket process of collecting labeled fingerprints at spatially fine-grained locations to form the training dataset imposes a high overhead. Thus, the existing studies only target room-level localization [45,48] or recognize a limited number of locations (11 closed locations in [52]).…”