A Framework for Optimizing Co-adaptation in Body-Machine Interfaces

Abstract

The operation of a human-machine interface is increasingly often referred to as a two-learners problem, where both the human and the interface independently adapt their behavior based on shared information to improve joint performance over a specific task. Drawing inspiration from the field of body-machine interfaces, we take a different perspective and propose a framework for studying co-adaptation in scenarios where the evolution of the interface is dependent on the users' behavior and that do not require task goals to be explicitly defined. Our mathematical description of co-adaptation is built upon the assumption that the interface and the user agents co-adapt toward maximizing the interaction efficiency rather than optimizing task performance. This work describes a mathematical framework for body-machine interfaces where a naïve user interacts with an adaptive interface. The interface, modeled as a linear map from a space with high dimension (the user input) to a lower dimensional feedback, acts as an adaptive “tool” whose goal is to minimize transmission loss following an unsupervised learning procedure and has no knowledge of the task being performed by the user. The user is modeled as a non-stationary multivariate Gaussian generative process that produces a sequence of actions that is either statistically independent or correlated. Dependent data is used to model the output of an action selection module concerned with achieving some unknown goal dictated by the task. The framework assumes that in parallel to this explicit objective, the user is implicitly learning a suitable but not necessarily optimal way to interact with the interface. Implicit learning is modeled as use-dependent learning modulated by a reward-based mechanism acting on the generative distribution. Through simulation, the work quantifies how the system evolves as a function of the learning time scales when a user learns to operate a static vs. an adaptive interface. We show that this novel framework can be directly exploited to readily simulate a variety of interaction scenarios, to facilitate the exploration of the parameters that lead to optimal learning dynamics of the joint system, and to provide an empirical proof for the superiority of human-machine co-adaptation over user adaptation.

Using Eye Gaze to Train an Adaptive Myoelectric Interface

“…The process involves recording and filtering raw sEMG activity, followed by the selection of relevant features [16]. These selected features are then mapped to low-dimensional control signals, commonly achieved through linear dimensionality reduction (e.g, principal component analysis or non-negative matrix factorization [13, 50]), regression [27, 41], or classification [48, 60]. These mappings in myoelectric interfaces are typically programmed offline, without the user in the loop [13].…”

An Exploratory Multi-Session Study of Learning High-Dimensional Body-Machine Interfacing for Assistive Robot Control

“…A classic approach to designing a BoMI map is to describe a linear relationship between sensor measurements and robot control commands [6]. More recent examples use iterative linear methods [14], feedback control (as opposed to feedforward control) [15], and deep learning methods such as adaptive nonlinear autoencoders [16].…”

Decoding of unimanual and bimanual reach-and-grasp actions from EMG and IMU signals in persons with cervical spinal cord injury

“…Previous experiments have shown that tree-based models such as Random Forests seem to perform well in a supervised multimodal HMI for activity recognition [24]. Additional factors such as a co-adaptive environment can further enhance HMI performance [30,31]. Hereby, human and machine interact through closed feedback loops in a mutual learning environment.…”

“…The user receives constant feedback based on his actions, while the machine receives new training samples over time and adapts to the human. This can potentially stabilize HMI performance by mitigating non-stationary properties and helps compensate for effects over time, such as fatigue and sensor signal degradation [31][32][33][34].…”