Fast Efficient Hyperparameter Tuning for Policy Gradients

Paul, Supratik; Kurin, Vitaly; Whiteson, Shimon

doi:10.48550/arxiv.1902.06583

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL training is notoriously sensitive to hyper-parameters and environment changes [18]. Recently, many works attempted to take techniques in AutoML to alleviate human intervention, for example, hyper-parameter optimization [7,36,49,53], reward search [9,45] and network architecture search [38,10]. In contrast to these methods which optimize a new configuration for each environment, we search for auxiliary loss functions that generalize across different settings such as (i) different robots of control; (ii) different data types of observation; (iii) partially observable settings; (iv) different network architectures; (v) different benchmark domains.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning with Automated Auxiliary Loss Search

He¹,

Zhang²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

A good state representation is crucial to solving complicated reinforcement learning (RL) challenges. Many recent works focus on designing auxiliary losses for learning informative representations. Unfortunately, these handcrafted objectives rely heavily on expert knowledge and may be sub-optimal. In this paper, we propose a principled and universal method for learning better representations with auxiliary loss functions, named Automated Auxiliary Loss Search (A2LS), which automatically searches for top-performing auxiliary loss functions for RL. Specifically, based on the collected trajectory data, we define a general auxiliary loss space of size 7.5 × 10 20 and explore the space with an efficient evolutionary search strategy. Empirical results show that the discovered auxiliary loss (namely, A2-winner) significantly improves the performance on both high-dimensional (image) and lowdimensional (vector) unseen tasks with much higher efficiency, showing promising generalization ability to different settings and even different benchmark domains. We conduct a statistical analysis to reveal the relations between patterns of auxiliary losses and RL performance. The codes and supplementary materials are available at https://seqml.github.io/a2ls.

show abstract

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning with Automated Auxiliary Loss Search

He¹,

Zhang²,

Ren³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…30 Hyperparameter tuning is selecting a set of optimal hyperparameters for a machine learning or CNN methods. 31 To evaluate the system's performance, this study considers Learning Rate (α), Batch Size (m), and Optimizers. Depending on the hyperparameter values, different results are obtained.…”

Section: Experimental Design Evaluation Metrics Evaluations Outcomes ...mentioning

confidence: 99%

A face mask detection system: An approach to fight with COVID‐19 scenario

Jayaswal

Dixit²

2022

Concurrency and Computation

View full text Add to dashboard Cite

A new coronavirus has caused a pandemic crisis around the globe. According to the WHO, this is an infectious illness that spreads from person to person. Therefore, the only way to avoid this infection is to take precautions. Wearing a mask is the most critical COVID-19 protection method because it prevents the virus from spreading from an infected person to a healthy one. This study reflects a deep learning method to create a system for detecting Face Masks. The paper proposes a unique FMDRT (Face Mask Dataset in Real-Time) dataset to determine whether a person is wearing a mask or not. The RFMD and Face Mask datasets are also taken from the internet to evaluate the performance of the proposed method. The CLAHE preprocessing method is employed to enhance the image quality, then resizing and Image augmentation techniques are used to convert it into a standard format and increase the size of the dataset, respectively. The pretrained Caffe face detector model is used to detect the faces, and then the lightweight transfer learning-based Xception model is applied for the feature extraction process. This paper recommended a novel model that is, CL-SSDXcept to distinguish the Face Mask or no mask images. However, accession with the MobileNetV2, VGG16, VGG19, and InceptionV3 models with different hyperparameter settings has been tested on the FMDRT dataset. We have also compared the results of the synthesized dataset FMDRT to the existing Face Mask datasets. The experimental results attained 98% test accuracy on the suggested dataset 'FMDRT' using the CL-SSDXcept method. The empirical findings have been reported at 50 iterations with tuned hyperparameter values with an average accuracy 98% and a loss of 0.05.

show abstract

“…OMPAC [10] uses a genetic algorithm to select the policy's softmax temperature and TD(λ) parameters in discrete environments such as Atari and Tetris. HOOF [37] generates a population of policy gradient updates with different loss function parameters and selects the best combination to continue training with weighted importance sampling. Agent57 [2] uses hyperparameter selection by a multi-armed bandit to improve exploration and surpasses human performance on the Atari benchmark.…”

Section: Related Workmentioning

confidence: 99%

Towards Automatic Actor-Critic Solutions to Continuous Control

Grigsby¹,

Yoo²,

Qi³

2021

Preprint

View full text Add to dashboard Cite

Model-free off-policy actor-critic methods are an efficient solution to complex continuous control tasks. However, these algorithms rely on a number of design tricks and many hyperparameters, making their applications to new domains difficult and computationally expensive. This paper creates an evolutionary approach that automatically tunes these design decisions and eliminates the RL-specific hyperparameters from the Soft Actor-Critic algorithm. Our design is sample efficient and provides practical advantages over baseline approaches, including improved exploration, generalization over multiple control frequencies, and a robust ensemble of high-performance policies. Empirically, we show that our agent outperforms well-tuned hyperparameter settings in popular benchmarks from the DeepMind Control Suite. We then apply it to new control tasks to find high-performance solutions with minimal compute and research effort.Preprint. Under review.

show abstract

Fast Efficient Hyperparameter Tuning for Policy Gradients

Cited by 4 publications

References 0 publications

Reinforcement Learning with Automated Auxiliary Loss Search

Reinforcement Learning with Automated Auxiliary Loss Search

A face mask detection system: An approach to fight with COVID‐19 scenario

Towards Automatic Actor-Critic Solutions to Continuous Control

Contact Info

Product

Resources

About