Optimal Control

Aschepkov, Leonid T.; Dolgy, Dmitriy V.; Kim, Taekyun; Agarwal, Ravi

doi:10.1007/978-3-319-49781-5

Cited by 15 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we obtain the optimality conditions of ML-POSC by employing Pontryagin's minimum principle [22][23][24][25] on the probability density function space (Figure 2 (bottom right)). The conventional approach in ML-POSC [14] and MFSC [30,31] can be interpreted as a conversion from Bellman's dynamic programming principle (Figure 2 (top right)) to Pontryagin's minimum principle (Figure 2 (bottom right)) on the probability density function space.…”

Section: Pontryagin's Minimum Principlementioning

confidence: 99%

“…In this section, we briefly review Pontryagin's minimum principle in deterministic control [22][23][24][25].…”

Section: Fundingmentioning

confidence: 99%

“…In this subsection, we formulate deterministic control [22][23][24][25]. The state of the system s t ∈ R d s at time t ∈ [0, T] evolves according to the following ordinary differential equation (ODE):…”

Section: Fundingmentioning

confidence: 99%

“…To propose an algorithm to solve ML-POSC, we first show that the system of HJB-FP equations can be interpreted via Pontryagin's minimum principle on the probability density function space. Pontryagin's minimum principle is one of the most representative approaches to the deterministic optimal control problem, which converts it into the twopoint boundary value problem of the forward state equation and the backward adjoint equation [22][23][24][25]. We formally show that the system of HJB-FP equations is an extension of the system of adjoint and state equations from the deterministic optimal control problem to the stochastic optimal control problem.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori

Kobayashi

2023

Entropy

View full text Add to dashboard Cite

Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker–Planck (FP) equation and the backward Hamilton–Jacobi–Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin’s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin’s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC.

show abstract

Section: Pontryagin's Minimum Principlementioning

confidence: 99%

“…In this section, we briefly review Pontryagin's minimum principle in deterministic control [22][23][24][25].…”

Section: Fundingmentioning

confidence: 99%

Section: Fundingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori

Kobayashi

2023

Entropy

View full text Add to dashboard Cite

show abstract

“…In fact, the ubiquitous use of state estimators (such as Kalman Filters) are, in fact, optimal solutions in the least-square sense [172]. In addition, this emphasis on optimality is central to Optimal Control [165] in its various modern forms as LQR, LQG, iLQR, iLQG (e.g., [173].). From its inception, however, the emphasis on optimality has proven problematic to stability margins, which has led to other forms of control such as robust control, path integral control, model predictive control, etc.…”

Section: Feasible Rather Than Optimal Functionmentioning

confidence: 99%

On neuromechanical approaches for the study of biological and robotic grasp and manipulation

Valero-Cuevas

Santello

2017

J NeuroEngineering Rehabil

View full text Add to dashboard Cite

Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank and open-minded assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas at the interface of neuromechanics, neuroscience, rehabilitation and robotics.

show abstract