Harmonic-Based Optimal Motion Planning in Constrained Workspaces Using Reinforcement Learning

Reactive optimal motion planning for a class of holonomic planar agents using reinforcement learning with provable guarantees

“…Nonetheless, note that, even though path length is a valid metric for these methods, the cost function (2) can not be directly evaluated, as they only produce feasible paths and not inputs. In previous works ( Rousseas et al, 2020 ; Rousseas et al, 2021 ; Rousseas et al, 2022 ), we have employed RRT ⋆ planners that include input-space sampling. However, due to sampling over a 4D space, the RRT ⋆ results presented therein do not consist of fair evaluations against the current method.…”

“…A notable class of APFs, namely, Artifical Harmonic Potential Fields (AHPFs) ( Loizou, 2011 ) have been employed in disk worlds, owing to lack of local minima inside their domain. Their limitations have also been treated through transformations of arbitrary workspaces into disk ones ( Vlantis et al, 2018a ), and the formulation of harmonic vector basis’ directly on the physical workspace ( Kim and Khosla, 1991 ; Rousseas et al, 2021 ; Rousseas et al, 2022 ). The above methods are consistent with the reactive approach of CT, through the formulation of a continuous velocity vector field over the entire workspace of a robot, while providing convergence and safety guarantees for the resulting velocity field.…”

An Efficient Solution to Optimal Motion Planning With Provable Safety and Convergence

“…Nevertheless, such methods are applied in the context of disk-worlds, thus, extending AHPFs to arbitrary workspaces, requires homeomorphic maps of arbitrary connected sets into such disk worlds [19], [20], [21]. More recently, the need for such homeomorphisms has been nullified through defining harmonic potentials over the physical workspace directly [22], [23], [24].…”

“…However, continuous OMP lacks solutions as widely adopted as SBMs, with only few works [25], [26], [27] that require solving a hard, non-linear Partial Differential Equation (PDE), or extensive parameter tuning. The authors have thus far focused on a mixed approach based on Policy Iteration (PI) [28], [22], [29], leveraging the properties of AHPFs, limiting however the space of possible policies. Our most recent works [30], [31], concentrate on a parameter-free solution, asymptotically yielding a globally optimal solution and overcoming the limitations of parametrizing the policy through AHPFs.…”

“…The authors have thus far focused on a mixed approach based on Policy Iteration (PI) [28], [22], [29], leveraging the properties of AHPFs, limiting however the space of possible policies. Our most recent works [30], [31], concentrate on a parameter-free solution, asymptotically yielding a globally optimal solution and overcoming the limitations of parametrizing the policy through AHPFs. Thus, solutions were extended to more complex dynamics [30] and more crucially, the Topological Perplexity (TP) [32] aspect of optimal control on multiply connected sets, which is vital for optimality, was adressed in [33].…”