Ubiquitous Writer: Robust Text Input for Small Mobile Devices via Acoustic Sensing

Yin, Huanpu; Zhou, Anfu; Liu, Liang; Wang, Ning; Ma, Huadóng

doi:10.1109/jiot.2019.2900355

Cited by 20 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SoundWave II adds two threshold values and exploits the Mel frequency cepstral coefficient (MFCC) to extract stable features and improve noise tolerance. UbiWriter [104] utilizes the microphones built in smartphone to record the audio signal generated by handwriting and analyzes the audio signal to realize the text input.…”

Section: A Passive Acoustic Sensingmentioning

confidence: 99%

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

et al. 2019

View full text Add to dashboard Cite

With the rapid development of Internet of Things, hand gesture recognition has drawn wide attention in the field of ubiquitous computing because it provides us with simple and natural human-computer interaction mode. Among these various implementations, hand gesture recognition using ultrasonic signals of smartphone has become a hot research topic due to its various advantages. In this paper, we consider the smartphone as an active sonar sensing system to identify hand movements. Specifically, the speakers emit ultrasonic signal and the microphone on the same phone receives the changed echo affected by hand movements. This paper investigates the state-of-the-art hand gesture applications and presents a comprehensive survey on the characteristics of studies using the active sonar sensing system. Firstly, we review the existing research of hand gesture recognition based on acoustic signals. After that, we introduce the characteristics of ultrasonic signal and describe the fundamental principle of hand gesture recognition. Then, we focus on the typical methods used in these studies and present a detailed analysis on signal generation, feature extraction, preprocessing, and recognition methods. Next, we investigate the state-of-the-art ultrasonic-based applications of hand gesture recognition using smartphone and analyze them in detail from dynamic gesture recognition and hand tracking. Afterwards, we make a discussion about these systems from signal acquisition, signal processing, and performance evaluation to obtain some insight into development of the ultrasonic hand gesture recognition system. Finally, we conclude by discussing the challenges, insight, and open issues involved in hand gesture recognition based on ultrasonic signal of the smartphone.

show abstract

Section: A Passive Acoustic Sensingmentioning

confidence: 99%

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Some of these objects are recording our activities, such as the touch screens of smartphones or tablets. Thus, adding sensory perception to objects and surfaces that surround us daily is an active area of research, and it has generated keen interest within the interactive surface community [ 1 , 2 , 3 , 4 ].…”

Section: Introductionmentioning

confidence: 99%

An Acoustic Sensing Gesture Recognition System Design Based on a Hidden Markov Model

Moreira

Perkusich

Luiz

2020

Sensors

View full text Add to dashboard Cite

Many human activities are tactile. Recognizing how a person touches an object or a surface surrounding them is an active area of research and it has generated keen interest within the interactive surface community. In this paper, we compare two machine learning techniques, namely Artificial Neural Network (ANN) and Hidden Markov Models (HMM), as they are some of the most common techniques with low computational cost used to classify an acoustic-based input. We employ a small and low-cost hardware design composed of a microphone, a stethoscope, a conditioning circuit, and a microcontroller. Together with an appropriate surface, we integrated these components into a passive gesture recognition input system for experimental evaluation. To perform the evaluation, we acquire the signals using a small microphone and send it through the microcontroller to MATLAB’s toolboxes to implement and evaluate the ANN and HMM models. We also present the hardware and software implementation and discuss the advantages and limitations of these techniques in gesture recognition while using a simple alphabet of three geometrical figures: circle, square, and triangle. The results validate the robustness of the HMM technique that achieved a success rate of 90%, with a shorter training time than the ANN.

show abstract

“…SoundWrite [19] also uses the collected acoustic signal to recognize the user's input gesture. UbiWriter [20] leverages small mobile devices for recognizing freestyle handwriting via acoustic sensing.…”

Section: Introductionmentioning

confidence: 99%

HCI on the Table: Robust Gesture Recognition Using Acoustic Sensing in Your Hand

et al. 2020

View full text Add to dashboard Cite

The paper proposes a new HCI mechanism for device-free gesture recognition on the table using acoustic signal, which can extend the gesture input and interactions beyond the tiny screen of mobile device and allow users to provide input without blocking screen view. Previous researches have either relied on additional devices (e.g., special wearable device and mouse) or required active acoustic signals which demand additional cost and be less prone to popularize, while we explore the device-free gesture recognition using passive acoustic signals. This technology is more challenging due to the lack of an effective approach to eliminate the inherent ambient noise disturbances and extract stable gesture features. We fuse both short time energy (STE) and zero-crossing rate (ZCR) to identify the effective signals from the original input, and leverage the Mel frequency cepstral coefficients (MFCC), cochlear filter cepstral coefficients (CFCC) to extract the stable features from different gestures. The unique features in support vector machine (SVM) classifier achieve a high gesture recognition accuracy from the noisy scenarios and mismatched conditions. Implementation on the Android system has realized real-time processing of the feature extraction and gesture recognition. Extensive evaluations show our algorithm has a better noise tolerant performance and the system could recognize seven common gestures (click, flip left/right, scroll up/down, zoom in/out) on smart devices with an accuracy of 93.2%.

show abstract

Ubiquitous Writer: Robust Text Input for Small Mobile Devices via Acoustic Sensing

Cited by 20 publications

References 31 publications

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

Hand Gesture Recognition Based on Active Ultrasonic Sensing of Smartphone: A Survey

An Acoustic Sensing Gesture Recognition System Design Based on a Hidden Markov Model

HCI on the Table: Robust Gesture Recognition Using Acoustic Sensing in Your Hand

Contact Info

Product

Resources

About