Analysis of Zero-Shot Crosslingual Learning between English and Korean for Named Entity Recognition

Kim, Jong-In; Choi, Nayoung; Lim, Sanghyeok; Kim, Jungwhan; Soo-Jin, Chung; Woo, Hyun-Soo; Song, Min Jae; Choi, Jinho D.

doi:10.18653/v1/2021.mrl-1.19

Cited by 2 publications

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The information bottleneck (IB) was first proposed in [30]. The principle of IB is to learn a good representation that can retain task-relevance information while reducing taskirrelevance information [17,2]. Since IB is plagued by computational mutual information, many methods are devoted to finding its lower bound.…”

Section: Related Workmentioning

confidence: 99%

Improving Visual Reinforcement Learning with Discrete Information Bottleneck Approach

Wang,

2023

Frontiers in Artificial Intelligence and Applications

View full text Add to dashboard Cite

Contrastive learning has been used to learn useful low-dimensional state representations in visual reinforcement learning (RL). Such state representations substantially improve the sample efficiency of visual RL. Nevertheless, existing contrastive learning-based RL methods have the problem of unstable training. Such instability comes from the fact that contrastive learning requires an extremely large batch size (e.g., 4096 or larger), while current contrastive learning-based RL methods typically set a small batch size (e.g., 512). In this paper, we propose an approach of discrete information bottleneck (DIB) to address this problem. DIB applies the technique of discretization and information bottleneck to contrastive learning in representing the state with concise discrete representation. Using this discrete representation for policy learning results in more stable algorithm training and higher sample efficiency with a small batch size. We demonstrate the advantage of discrete state representation of DIB on several continuous control tasks in the DeepMind Control suite. In the experiments, DIB outperforms prior visual RL methods, both model-based and model-free, in terms of performance and sample efficiency.

show abstract

Section: Related Workmentioning

confidence: 99%