Vibration-Based Surface Recognition for Smartphones

Abstract

With various sensors in a smartphone, it is now possible to obtain information about a user and her surroundings, such as the location of a smartphone and the activity of the smartphone user, and the obtained context information is being used to provide new services to the users. In this paper, we propose VibePhone, which uses a built-in vibrator and accelerometer, for recognizing the type of surfaces contacted by a smartphone, enabling the sense of touch in smartphones. For humans and animals, the sense of touch is fundamental for both recognizing and learning the properties of objects. The sense of touch is obtained from the texture of an object and humans recognize the type of an object by scrubbing the surface with fingers. Since a smartphone cannot physically scrub the contacting surface, we emulate the touch by generating vibrations using a smartphone and propose a method to recognize the type of contacting objects. The recognition of the object type by vibration alone is an extremely difficult task, even for a human. However, we demonstrate that it is possible to distinguish object types into broad categories where a phone is usually placed, e.g., sofas, plastic tables, wooden tables, hands, backpacks, and pants pockets. The proposed VibePhone system achieves an accuracy over 85% on average. We have prototyped VibePhone on an Android-based smartphone which changes its background display based on the contacting surface. We envision that the haptic perception in future smartphones will enable new experiences to the users.

VibroTactor

“…A small but useful category of two places (soft and hard materials) can be distinguished in excess of 99% accuracy. We believe this is an excellent result considering the results of previous works [2,4]. In addition, we found that the vibration echoes change depending on the place within the same location.…”

“…This technique offers reliable and fast (or real-time) localization, but one disadvantage is that such hardware is still not supported in commercially available mobile devices. Given this background, some researchers have investigated the inertial sensors of mobile devices, such as their microphones or accelerometers [2,4]. Cho et al [2] introduced a surface recognition technique that used a built-in vibration motor and accelerometer.…”

“…Given this background, some researchers have investigated the inertial sensors of mobile devices, such as their microphones or accelerometers [2,4]. Cho et al [2] introduced a surface recognition technique that used a built-in vibration motor and accelerometer. They used the vibration signature in conjunction with the inertial accelerometer in their device.…”

Recognizing Falls and Surfaces Using Mobile Devices

“…On both works, there is no analysis connecting the device context and surface with falls detection. Moreover, [Cho et al 2012] shows the "VibePhone" solution, that focuses on the same concepts as the previous works. An 44º SEMISH -Seminário Integrado de Software e Hardware SVM (Support Vector Machine) is used to classify vibration data obtained from an accelerometer that varies according to the surface where a smartphone is placed.…”

“…Although imperative, knowing when a device/user is falling is not enough by itself without having surface dangerousness estimation. Another set of solutions tries to infer the surface characteristics where a device is placed [Darbar and Samanta 2015] [Cho et al 2012].…”