Noncontact human-machine interaction for air-writing with ultrasound technology

Abstract

With rapid development of internet of things and virtual reality (VR), sensors based on human-machine interaction (HMI) are crucially required to have attributes of natural and intuitive operation experience using various adaptable and hygienic noncontact techniques. In this study, we develop a real-time noncontact HMI system for air-writing based on piezoelectric micromachined ultrasonic transducers (pMUTs), which have great advantages including easy integration, miniaturization, low power consumption, and less influences by environmental factors such as light or sound. However, pMUTs have problems of weak signals and limited measurement range for complex gesture recognitions. To address these issues, we propose a machine learning algorithm which effectively fuses multiple features of time-of-flight, voltage amplitude, and echo energy and significantly increase the detection range and accuracy of pMUTs for arbitrary gesture recognition. We demonstrate real-time computer control to search and browse websites simply using only one finger air-writing with a recognition accuracy of 96.62% for 16 gestures of characters (including numbers, letters, and symbols) within a distance range of 15 cm. We believe our method revolutionize functionalities and adaptabilities of noncontact HMI in VR, smart home and vehicle, smart cities, and healthcare.