AR‐Enabled Persistent Human–Machine Interfaces via a Scalable Soft Electrode Array

Abstract

Augmented reality (AR) is a computer graphics technique that creates a seamless interface between the real and virtual worlds. AR usage rapidly spreads across diverse areas, such as healthcare, education, and entertainment. Despite its immense potential, AR interface controls rely on an external joystick, a smartphone, or a fixed camera system susceptible to lighting. Here, an AR‐integrated soft wearable electronic system that detects the gestures of a subject for more intuitive, accurate, and direct control of external systems is introduced. Specifically, a soft, all‐in‐one wearable device includes a scalable electrode array and integrated wireless system to measure electromyograms for real‐time continuous recognition of hand gestures. An advanced machine learning algorithm embedded in the system enables the classification of ten different classes with an accuracy of 96.08%. Compared to the conventional rigid wearables, the multi‐channel soft wearable system offers an enhanced signal‐to‐noise ratio and consistency over multiple uses due to skin conformality. The demonstration of the AR‐integrated soft wearable system for drone control captures the potential of the platform technology to offer numerous human–machine interface opportunities for users to interact remotely with external hardware and software.