On the Impact of Gravity Compensation on Reinforcement Learning in Goal-Reaching Tasks for Robotic Manipulators

Abstract

Advances in machine learning technologies in recent years have facilitated developments in autonomous robotic systems. Designing these autonomous systems typically requires manually specified models of the robotic system and world when using classical control-based strategies, or time consuming and computationally expensive data-driven training when using learning-based strategies. Combination of classical control and learning-based strategies may mitigate both requirements. However, the performance of the combined control system is not obvious given that there are two separate controllers. This paper focuses on one such combination, which uses gravity-compensation together with reinforcement learning (RL). We present a study of the effects of gravity compensation on the performance of two reinforcement learning algorithms when solving reaching tasks using a simulated seven-degree-of-freedom robotic arm. The results of our study demonstrate that gravity compensation coupled with RL can reduce the training required in reaching tasks involving elevated target locations, but not all target locations.

Aplicações de Aprendizado por Reforço em Manipuladores Robóticos: Uma Revisão Sistemática

“…Estas abordagens foram: aprendizado de movimentos ponto a ponto [3], aprendizado com feedback interativo [27], algoritmos de interac ¸ão contínua [24], manipulac ¸ão de objetos [35], tarefa de abertura de porta [45], manipulac ¸ão coordenada de multi-robôs [20], controle neural adaptativo [42], controle de manipuladores [18], planejamento de trajetória [47], inspec ¸ão robótica [14] e controle de posic ¸ão [49]. [48,17,19,38] Não utilizou [3,27,24,35,45,20,42,18,47,14,49] Verifica-se também que a variedade de manipuladores robóticos utilizados é ampla, sendo o modelo UR3 da Universal Robots o mais utilizado entre estes em trabalhos com enfoque em: Tarefa peg-in-hole [6,4] e controle de brac ¸o duplo robótico [23]; seguido do PANDA [10,34], UR5 [31,22], RM-X52 [32,33] e IRB 1600 [1,2]. Além disso, 3 trabalhos fizeram o uso de manipuladores produzidos em laboratório, customizados ou com pec ¸as impressas em 3D, implementados em: Controle de articulac ¸ões robóticas [36], planejamento de movimento [46] e mapeamento de controlador de brac ¸o robótico [37].…”

“…O estudo de [10] fez uma comparac ¸ão entre os resultados obtidos com dois algoritmos: Actor Critic Using Kronecker-Factored Trust Region (ACKTR) e Proximal Policy Optimization (PPO2), os quais foram aplicados no estudo do impacto da compensac ¸ão da gravidade nas tarefas de alcance para manipuladores robóticos. E apesar de ter uma variac ¸ão significativamente maior entre cada sessão de treinamento, PPO2 conseguiu treinar 5 vezes mais rápido do que o ACKTR.…”

“…Finalmente foi demonstrado, que as taxas de aprendizado decrescentes superaram as taxas de aprendizado constantes. [10] realizou 192 permutac ¸ões de selec ¸ões de hiperparâmetros para o algoritimo ACKTR e 128 para o algorítimo PPO2, nas quais foram realizadas dez sessões de treinamento em cada uma delas. Para o ACKTR, foi utilizado 2, 5x10 5 interavalos de tempo (passos dados no ambiente), enquanto para o PPO2, 1x10 5 intervalos de tempo.…”

Skill Fusion in Hybrid Robotic Framework for Visual Object Goal Navigation

“…In recent years, an orthogonal approach, based on end-to-end machine learning, became widespread. State-of-the-Art learnable methods are not only capable of navigating freely through complex scenes without explicit map reconstruction [1,8,9], but also have abilities for manipulation [10], long-horizon planning [11], building world models [12], and communicating with humans through natural languages [13]. The success of these methods comes mainly from the high amount of data used for training.…”

The Study of the Dynamics of a Cebâşev Manipulator

“…Dynamic and friction parameters of an industrial robot with repeatability identification, comparison and analysis are other important aspects of dynamic and robotic processes in the industry (Hao et al, 2021). The impact of gravity compensation on reinforcement learning in goalsetting tasks for robotic manipulators is a relatively new problem in dynamic disciplines (Fugal et al, 2021). Another dynamic new aspect is the mechatronic redesign of a manual assembly workstation in collaboration with wiring assemblies (Palomba et al, 2021), which can be directly associated with new technological processes.…”