Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits

Abstract

Understanding users' gait preferences of a lowerbody exoskeleton requires optimizing over the high-dimensional gait parameter space. However, existing preference-based learning methods have only explored low-dimensional domains due to computational limitations. To learn user preferences in high dimensions, this work presents LINECOSPAR, a human-inthe-loop preference-based framework that enables optimization over many parameters by iteratively exploring one-dimensional subspaces. Additionally, this work identifies gait attributes that characterize broader preferences across users. In simulations and human trials, we empirically verify that LINECOSPAR is a sample-efficient approach for high-dimensional preference optimization. Our analysis of the experimental data reveals a correspondence between human preferences and objective measures of dynamic stability, while also highlighting inconsistencies in the utility functions underlying different users' gait preferences. This has implications for exoskeleton gait synthesis, an active field with applications to clinical use and patient rehabilitation.

Review of control strategies for lower-limb exoskeletons to assist gait

“…In many cases, the trajectories are significantly changed or fully generated at runtime, and some papers are completely dedicated to the problem of optimization/ generation of trajectories [190][191][192][193]. In some studies, model-based computations [194][195][196][197] or polynomial minimum jerk trajectory generation methods [94] have been used to generate the trajectories offline.…”

ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes

“…Common choices of link function (g p and g o ) include the Gaussian cumulative distribution function [17], [19] and the sigmoid function, g(x) = (1 + e −x ) −1 [7]. We model feedback via the sigmoid link function because empirical results suggest that a heavier-tailed noise distribution improves performance.…”

“…Most existing work in high-dimensional Gaussian process learning requires quantitative feedback [21], [22]. Previous work in preference-based high-dimensional Gaussian process learning [7] models the posterior over a sequence of onedimensional subspaces. However, this approach applies only to the regret minimization problem because each onedimensional subspace includes the action maximizing the posterior.…”

“…Given the need to maximize sample efficiency from limited trials, each choice of goal implies a different sampling strategy for data collection. Previous work [6], [7] focused on the first goal of direct function optimization, but their approach did not reliably learn the entire utility landscape governing user preferences across gaits. Thus, we propose an alternative approach aiming for the second goal, to characterize the entire landscape (albeit with less finegrained data in the region close to the optimal gait).…”

Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots

“…As opposed to relying on just one field, this work explores combining the successes of both: the formality of stability from control theory and the ability to learn the relationship between complex parameter combinations and their resulting locomotive behavior from machine learning. This is accomplished by building upon our previous results [24], [25] and systematically integrating preference-based learning with gait generation via HZD optimization. The result is optimal walking on hardware based only on relative pairwise preferences from the human operator (i.e.…”