Human–Machine Interaction via Dual Modes of Voice and Gesture Enabled by Triboelectric Nanogenerator and Machine Learning

Abstract

With the development of science and technology, human− machine interaction has brought great benefits to the society. Here, we design a voice and gesture signal translator (VGST), which can translate natural actions into electrical signals and realize efficient communication in human−machine interface. By spraying silk protein on the copper of the device, the VGST can achieve improved output and a wide frequency response of 20−2000 Hz with a high sensitivity of 167 mV/dB, and the resolution of frequency detection can reach 0.1 Hz. By designing its internal structure, its resonant frequency and output voltage can be adjusted. The VGST can be used as a high-fidelity platform to effectively recover recorded music and can also be combined with machine learning algorithms to realize the function of speech recognition with a high accuracy rate of 97%. It also has good antinoise performance to recognize speech correctly even in noisy environments. Meanwhile, in gesture recognition, the triboelectric translator is able to recognize simple hand gestures and to judge the distance between hand and the VGST based on the principle of electrostatic induction. This work demonstrates that triboelectric nanogenerator (TENG) technology can have great application prospects and significant advantages in human−machine interaction and high-fidelity platforms.

Triboelectric Nanogenerator Assisted by Machine Learning

“…Therefore, creating a more humanized and intuitive human–computer interaction coenergy is a problem of great practical significance. Luo et al designed a voice and gesture signal translator (VGST) to improve the efficiency of human-machine interface communication (Figure a–c). The frequency response range of the signal converter is 20–2000 Hz, the sensitivity is 167 mV/dB, the frequency detection resolution is up to 0.1 Hz, and the natural action is successfully converted into an electrical signal.…”

“…(d, e) VGST combines machine learning techniques and its recognition accuracy. (f) Test of the VGST’s sound recording function …”

Roadmap to Human–Machine Interaction through Triboelectric Nanogenerator and Machine Learning Convergence

“…The electrical output generation in the TENG can be evaluated by the following equations Q normals normalc = S σ x ( t ) d 0 + x ( t ) V normalO normalC = σ x ( t ) ε where Q sc is the short circuit charge transfer; S is the surface area; σ is the surface charge density; x ( t ) is the gap distance as a function of time; V OC is the open circuit voltage; ε is the relative permittivity; and d 0 is the effective thickness ( d 0 = prefix∑ i = 1 n d ε i ε r i ). TENGs utilized the contact electrification effect to convert mechanical energy (such as vibrations, friction, or pressure) into electrical energy. , Figure illustrates the various operating modes of TENGs.…”

Recent Advances in Self-powered Sensors Based on Nanogenerators: From Material and Structural Design to Cutting-Edge Sensing Applications

“…When an external stimulus acts on the friction electric sensor, it changes the contact state between the surfaces of the friction material changes. This in turn leads to the generation and distribution of electrostatic charge, which is eventually reflected in the voltage signal to complete the sensing process. ,, Figure shows that there are four common operating modes of TENG: contact separation (CS), lateral sliding (LS), single electrode (SE), and freestanding (FS). The availability of various operating modes enables the utilization of friction-electric self-powered sensors in a diverse array of applications.…”