Using Human Ratings for Feedback Control: A Supervised Learning Approach With Application to Rehabilitation Robotics

Abstract

This paper presents a method for tailoring a parametric controller based on human ratings. The method leverages supervised learning concepts in order to train a reward model from data. It is applied to a gait rehabilitation robot with the goal of teaching the robot how to walk patients physiologically. In this context, the reward model judges the physiology of the gait cycle (instead of therapists) using sensor measurements provided by the robot and the automatic feedback controller chooses the input settings of the robot so as to maximize the reward. The key advantage of the proposed method is that only a few input adaptations are necessary to achieve a physiological gait cycle. Experiments with non-disabled subjects show that the proposed method permits the incorporation of human expertise into a control law and to automatically walk patients physiologically.

Inverse Learning for Data-Driven Calibration of Model-Based Statistical Path Planning

“…Further related-but conceptually different-approaches are to learn policies rather than an objective function [38], [39], which is often referred to as imitation learning, or to use labeled data in order to learn an objective function, e.g., using supervised learning [40]- [42]. Notably, [40] uses semisupervised learning with a similar motivation, where drivers are classified into aggressive and normal driving styles based on a few labeled data points.…”

“…Therefore, we are more constrained in the solution but need less data and have properties that are invariant through the learning process, which we can use to enforce safe behaviors while learning. Compared to supervised learning methods [40]- [42], we do not require labeled data in order to learn a control objective. On the other hand, inverse learning methods that use unlabeled data, such as IOC, IRL, and our method, require the assumption that the data represent desirable behavior.…”

Distributed Personalized Gradient Tracking with Convex Parametric Models

“…2. The estimates Pi,t , qi,t and ri,t of the unknown parameters P i , q i and r i of U i are updated by means of an ad-hoc learning procedure (7). Such a procedure relies on a recursive least square scheme which makes use only of the most updated data (y i,t , x i,t ), thus not requiring to store and use all the past points generated by the distributed algorithm.…”

“…The aim of the learning part of Algorithm 1 (cf. (7)) is to provide a recursive scheme to let each agent i estimate the unknown parameters of U i . Specifically, the considered scheme aims at solving, for each t, the least squares (LS) problem minimize…”

“…Important examples of this class of systems are the energy grid, whenever human preferences are considered into energy demands and consumption schedules [3,4], or transportation infrastructures, whenever human behavior and routing preferences are taken into account in the decision making process [5,6]. Other examples stem from personalized healthcare [7] and robotics [8].…”