Computational Optimization of Image-Based Reinforcement Learning for Robotics

Abstract

The robotics field has been deeply influenced by the advent of deep learning. In recent years, this trend has been characterized by the adoption of large, pretrained models for robotic use cases, which are not compatible with the computational hardware available in robotic systems. Moreover, such large, computationally intensive models impede the low-latency execution which is required for many closed-loop control systems. In this work, we propose different strategies for improving the computational efficiency of the deep-learning models adopted in reinforcement-learning (RL) scenarios. As a use-case project, we consider an image-based RL method on the synergy between push-and-grasp actions. As a first optimization step, we reduce the model architecture in complexity, by decreasing the number of layers and by altering the architecture structure. Second, we consider downscaling the input resolution to reduce the computational load. Finally, we perform weight quantization, where we compare post-training quantization and quantized-aware training. We benchmark the improvements introduced in each optimization by running a standard testing routine. We show that the optimization strategies introduced can improve the computational efficiency by around 300 times, while also slightly improving the functional performance of the system. In addition, we demonstrate closed-loop control behaviour on a real-world robot, while processing everything on a Jetson Xavier NX edge device.

Slow but flexible or fast but rigid? Discrete and continuous processes compared

“…Whether an agent maintains a simple continuous generative model or makes use of a higher-level discrete model for planning, to operate in realistic scenarios it needs the notion of environmental beliefs . Maintaining additional beliefs over the environment has been used for actively inferring object-centric representations [41], and we here turn to the problem of tackling dynamic tasks that comprise multiple steps or composite movements. If the agent’s goal is to reach a moving object, it must maintain not only a best guess about its hand position, but also about the object position [42].…”

An Overview of Computational Coronary Physiology Technologies Based on Medical Imaging and Artificial Intelligence

“…This nuanced understanding is vital in comparing the processing speeds of QFR, OFR, CT-FFR, and iFR [ 39 ]. Deep learning models can perform a large number of matrix operations in parallel on GPUs, thereby improving the computational efficiency and accuracy [ 40 ]. However, some studies have also pointed out that the computational speed of QFR and OFR is also improving, and the results can be obtained within a few minutes by optimizing the algorithm and platform, and no additional equipment and software are required [ 41 ].…”

Symmetry and complexity in object-centric deep active inference models

“…In the equivariant representation, the transformation to the input observation is preserved and cascaded into the intermediate space. Equivariant representations have been highly adopted in the context of machine learning [15,24–26], and are considered crucial to achieving generalization features from the model at hand. The adoption of equivariant architectures is favoured in the research of ‘disentangled’ systems by the machine learning community where they have demonstrated their efficiency in generalization, imagination and abstraction reasoning.…”