A POMDP framework for human-in-the-loop system

Lam, Chi-Pang; Sastry, S. Shankar

doi:10.1109/cdc.2014.7040333

Cited by 20 publications

(16 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Except for focusing on energy waste, better feedback mechanisms and interact optimiza- [182] 2014 Universal robotic control Lam & Sastry et al [183] 2014 Ergonomics Gopinath et al [184] 2016 Shared autonomy Holzinger [185] 2016 Interactive system application Conor Walsh [186] 2018 Soft wearable robots Sutton et al [187] 2018 Trustworthiness mechanism Inoue et al [188] 2019 Weakly control Mello et al [189] 2019 Ergonomics Adolfo & Yu [190] 2020 Robotic sport control Fang and Yuan [191] 2020 Ergonomics Papallas et al [192] 2020 Group control Zhou et al [193] 2020 Privacy protection Paikens et al [194] 2020 Service industry Lee et al [195] 2020 Traffic control Usman et al [196] 2020 Traffic control Cicirelli et al [197] 2020 Home automatic Cimini et al [198] 2020 Production systems Gil et al [199] 2020 Interactive system theory Abraham et al [200] 2021 Shared autonomy Ciabattoni et al [201] 2021 Trustworthiness mechanism Fosch-Villaronga et al [202] 2021 Surgery automation tion can be another issue. Lam and Sastry [183] proposed a unified framework called partially observable Markov decision process(POMDP) for modeling the feedback of three components of SHI-HITL systems named human model, the observation model and the dynamic machine model. The proposed framework benefits from reasoning human internal state, handling observation errors, and balancing the feedback to humans and machines.…”

Section: Application2: System Optimizationmentioning

confidence: 99%

A Survey of Human-in-the-loop for Machine Learning

Wu,

Xiao,

Sun

et al. 2021

Preprint

View full text Add to dashboard Cite

Human-in-the-loop aims to train an accurate prediction model with minimum cost by integrating human knowledge and experience. Humans can provide training data for machine learning applications and directly accomplish some tasks that are hard for computers in the pipeline with the help of machine-based approaches. In this paper, we survey existing works on human-in-the-loop from a data perspective and classify them into three categories with a progressive relationship: (1) the work of improving model performance from data processing, (2) the work of improving model performance through interventional model training, and (3) the design of the system independent human-in-the-loop. Using the above categorization, we summarize major approaches in the field, along with their technical strengths/ weaknesses, we have simple classification and discussion in natural language processing, computer vision, and others. Besides, we provide some open challenges and opportunities. This survey intends to provide a high-level summarization for human-in-the-loop and motivates interested readers to consider approaches for designing effective human-in-the-loop solutions.

show abstract

Section: Application2: System Optimizationmentioning

confidence: 99%

A Survey of Human-in-the-loop for Machine Learning

Wu,

Xiao,

Sun

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The global minimizer of the unconstrained optimization, simply min f (u), is denoted by u * , while that of the constrained one, described by (8), is denoted by u † . Trivially, f (u * ) ≤ f (u † ) holds.…”

Section: A Problem Settingmentioning

confidence: 99%

“…Some works have tried to construct more general control frameworks for HIL systems in e.g. [7], [8], [9], [14], [15], [16], [17]. In [16], [17], humans are modeled as uncertainties vgupta2@nd.edu *This work was supported by CREST No.…”

Section: Introductionmentioning

confidence: 99%

“…In [7], [8], [9], [14], [15], humans are positively involved in the feedback loop of the controlled systems. In [7], [8], [9], humans are modeled as reference generators for autonomous controlled robots. This can be viewed as human decision-making being involved in the outer feedback loop of the overall control system.…”

Section: Introductionmentioning

confidence: 99%

“…Graphical interpretation of the input candidate U and decision by H. If U ∈ {U } ∆ρ is provided by the controller K, rational H chooses u † , which is the minimizer of the constrained optimization problem(8).…”

mentioning

confidence: 99%

See 2 more Smart Citations

“Weak” Control for Human-in-the-Loop Systems

Inoue

Gupta

2019

IEEE Control Syst. Lett.

View full text Add to dashboard Cite

In this letter, we propose a control framework for human-in-the-loop systems, in which many human decision makers are involved in the feedback loop composed of a plant and a controller. The novelty of the framework is that the decision makers are weakly controlled; in other words, they receive a set of admissible control actions from the controller and choose one of them in accordance with their private preferences. For example, the decision makers can decide their actions to minimize their own costs or by simply relying on their experience and intuition. A class of controllers which output set-valued signals is proposed, and it is shown that the overall control system is stable independently of the decisions made by the humans. Finally, a learning algorithm is applied to the controller that updates the controller parameters to reduce the achievable minimal costs for the decision makers. Effective use of the algorithm is demonstrated in a numerical experiment.

show abstract