Staircase Traversal via Reinforcement Learning for Active Reconfiguration of Assistive Robots

Abstract

Assistive robots introduce a new paradigm for developing advanced personalized services. At the same time, the variability and stochasticity of environments, hardware and unknown parameters of the interaction complicates their modelling, as in the case of staircase traversal. For this task, we propose to treat the problem of robot configuration control within a reinforcement learning framework, using policy gradient optimization. In particular, we examine the use of safety or traction measures as a means for endowing the learned policy with desired properties. Using the proposed framework, we present extensive qualitative and quantitative results where a simulated robot learns to negotiate staircases of variable size, while being subjected to different levels of sensing noise.

Staircase Negotiation Learning for Articulated Tracked Robots with Varying Degrees of Freedom

“…In our previous work [6] where we opted for independent tracks and flipper control, the robot was able to learn a behavior required for accomplishing staircase ascent while respecting safety constraints. However, with the inclusion of DOF of an articulated arm, separate control of the main tracks with the arms actuators can be sub-optimal.…”

“…In this section we present reward functions used for learning staircase ascent and descent. In detail, we employ the same positive reward as in [6] representing the total travelled distance on the stairs which drives learning, namely:…”

“…1) Center of Gravity stabilization: The center of gravity of the robotic platform is known to be instrumental in preventing tip-over accidents and improving stability [9], [19]. In [6] we enabled a robot to learn a stable behavior by favoring robot poses with low center of gravity (COG) with respect to the underlying surface A that represents the Normalized Energy Stability Margin (NESM) [20]. The presence of an arm nonetheless may place the COG low but also close to the "safety zone" border [9], without violating the NESM.…”

“…A recent approach for simulating robot tracks that provides more realistic robot track simulation and has shown highly plausible results and faster simulation is the Contact Surface Motion (CSM) [5], which we employ in the present work. Relatedly, we recently developed a RL-based framework for tracked robots equipped with flippers, for learning ascent control policies of staircases of varying size [6]. In continuation of that work, we present here our latest developments for learning more elaborate mobility skills that are altogether critical in enabling tracked robots to undertake the commonly required task of object search and transport.…”

“…Section II presents most relevant works that deal with the problem of staircase negotiation for articulated tracked robots. In section III we augment our previous formalization [6] to further allow learning of staircase descent and consider the influence of an articulated arm. Finally, in section IV the experimental setup is presented together with the obtained qualitative and quantitative results.…”