Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach

Jiang, Haoming; Dai, Bo; Yang, Mengjiao; Zhao, Tuo; Wei, Wei

doi:10.18653/v1/2021.emnlp-main.589

Cited by 3 publications

(1 citation statement)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An on-policy algorithm is based on a single policy, denoted as π, which is utilized by an agent to take actions in a given state s, aiming to obtain a reward [ 48 ]. In contrast, off-policy algorithms involve the use of two policies, the target policy and the behavior policy, denoted as π and μ, respectively [ 49 , 50 ]. The target policy is the one that the agent seeks to learn and optimize, while the behavior policy generates the observations that are used for learning.…”

Section: Reinforcement Learning (Rl)mentioning

confidence: 99%

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review

Al-Hamadani,

Fadhel,

Alzubaidi

et al. 2024

Sensors

View full text Add to dashboard Cite

Reinforcement learning (RL) has emerged as a dynamic and transformative paradigm in artificial intelligence, offering the promise of intelligent decision-making in complex and dynamic environments. This unique feature enables RL to address sequential decision-making problems with simultaneous sampling, evaluation, and feedback. As a result, RL techniques have become suitable candidates for developing powerful solutions in various domains. In this study, we present a comprehensive and systematic review of RL algorithms and applications. This review commences with an exploration of the foundations of RL and proceeds to examine each algorithm in detail, concluding with a comparative analysis of RL algorithms based on several criteria. This review then extends to two key applications of RL: robotics and healthcare. In robotics manipulation, RL enhances precision and adaptability in tasks such as object grasping and autonomous learning. In healthcare, this review turns its focus to the realm of cell growth problems, clarifying how RL has provided a data-driven approach for optimizing the growth of cell cultures and the development of therapeutic solutions. This review offers a comprehensive overview, shedding light on the evolving landscape of RL and its potential in two diverse yet interconnected fields.

show abstract

Section: Reinforcement Learning (Rl)mentioning

confidence: 99%

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review

Al-Hamadani,

Fadhel,

Alzubaidi

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

TRAIL: Near-Optimal Imitation Learning with Suboptimal Data

Yang¹,

Levine²,

Nachum³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The aim in imitation learning is to learn effective policies by utilizing near-optimal expert demonstrations. However, high-quality demonstrations from human experts can be expensive to obtain in large number. On the other hand, it is often much easier to obtain large quantities of suboptimal or task-agnostic trajectories, which are not useful for direct imitation, but can nevertheless provide insight into the dynamical structure of the environment, showing what could be done in the environment even if not what should be done. We ask the question, is it possible to utilize such suboptimal offline datasets to facilitate provably improved downstream imitation learning? In this work, we answer this question affirmatively and present training objectives that use offline datasets to learn a factored transition model whose structure enables the extraction of a latent action space. Our theoretical analysis shows that the learned latent action space can boost the sample-efficiency of downstream imitation learning, effectively reducing the need for large near-optimal expert datasets through the use of auxiliary non-expert data. To learn the latent action space in practice, we propose TRAIL (Transition-Reparametrized Actions for Imitation Learning), an algorithm that learns an energy-based transition model contrastively, and uses the transition model to reparametrize the action space for sample-efficient imitation learning. We evaluate the practicality of our objective through experiments on a set of navigation and locomotion tasks. Our results verify the benefits suggested by our theory and show that TRAIL is able to improve baseline imitation learning by up to 4x in performance. 1

show abstract

A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learning

Ren

Zhang

Szepesvári³

et al. 2021

Preprint

View full text Add to dashboard Cite

Representation learning lies at the heart of the empirical success of deep learning for dealing with the curse of dimensionality. However, the power of representation learning has not been fully exploited yet in reinforcement learning (RL), due to i), the trade-off between expressiveness and tractability; and ii), the coupling between exploration and representation learning. In this paper, we first reveal the fact that under some noise assumption in the stochastic control model, we can obtain the linear spectral feature of its corresponding Markov transition operator in closed-form for free. Based on this observation, we propose Spectral Dynamics Embedding (SPEDE), which breaks the trade-off and completes optimistic exploration for representation learning by exploiting the structure of the noise. We provide rigorous theoretical analysis of SPEDE, and demonstrate the practical superior performance over the existing state-of-the-art empirical algorithms on several benchmarks.

show abstract

Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach

Cited by 3 publications

References 54 publications

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review

Reinforcement Learning Algorithms and Applications in Healthcare and Robotics: A Comprehensive and Systematic Review

TRAIL: Near-Optimal Imitation Learning with Suboptimal Data

A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learning

Contact Info

Product

Resources

About