Plume Tracing via Model-Free Reinforcement Learning Method

Hu, Heng; Song, Shiji; Chen, C. L. Phillip

doi:10.1109/tnnls.2018.2885374

Cited by 57 publications

(23 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hu et al . [100] proposed a DRL‐based method for underwater plume tracking. The RL framework follows the actor‐critic structure, using the LSTM‐based deterministic policy gradient (DPG) algorithm to learn a tracking strategy.…”

Section: Learning Methodsmentioning

confidence: 99%

“…In Ref. [100], the reward function is adapted to the OSL background: positive rewards are gained if approaching the plume, while negative rewards are assigned if leaving the plume. Simulation results show that the proposed LSTM‐based DPG outweighs the standard DPG in efficiency and accuracy, and the resulted searching trajectory is a quite smooth one.…”

Section: Learning Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Recent Progress and Trend of Robot Odor Source Localization

Jing

Meng

Ishida

2021

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Accurate and prompt localization of hazardous release sources ensures a timely emergency response to potential damages. Robot odor source localization is the study of locating the release source based on mobile robots in real‐time, which has been experiencing rapid growth in recent years. This paper presents a review of this field, focusing on recent progress and development trends. Research works are grouped according to method principles, leading to four categories: reactive methods, heuristic searching methods, probabilistic inference methods, and learning methods. The odor‐source direction and distance prediction via in situ sensing and developments of simulators are also listed separately for discussion. Several outlooks are highlighted at the end. Along with the developments of related fields and practical needs, we believe this field is bound to keep on flourishing in the future. © 2021 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

show abstract

Section: Learning Methodsmentioning

confidence: 99%

Section: Learning Methodsmentioning

confidence: 99%

Recent Progress and Trend of Robot Odor Source Localization

Jing

Meng

Ishida

2021

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

show abstract

“…The key underlying challenge here is to process such large amounts of IoT data and perform efficient actions. Reinforcement Learning (RL) techniques [7][8][9] are attractive solutions for this challenge (to process IoT data) as RL approaches can learn and adapt to the environmental changes and can directly learn from historical data This Project was partially supported by the Department of National Defence's Innovation for Defence Excellence and Security (IDEaS) program, Canada.…”

Section: Introductionmentioning

confidence: 99%

“…Unlike supervised learning, RL approaches do not need labeled data, which is difficult to acquire in the context of SCs. Traditionally, RL algorithms are categorized into two main classes: (i) Model-Free (MF) methods [7][8][9], which learn the value function using sample trajectories, and; (ii) Model-Based (MB) methods [12] that estimate transition and reward functions through search trees or dynamic programming. Algorithms belonging to the former category (MF algorithms), typically, fail to rapidly adjust an agent to localized changes in the reward function.…”

Section: Introductionmentioning

confidence: 99%

Makf-Sr: Multi-Agent Adaptive Kalman Filtering-Based Successor Representations

Salimibeni

Malekzadeh

Mohammadi

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

The paper is motivated by the importance of the Smart Cities (SC) concept for future management of global urbanization and energy consumption. Multi-agent Reinforcement Learning (RL) is an efficient solution to utilize large amount of sensory data provided by the Internet of Things (IoT) infrastructure of the SCs for city-wide decision making and managing demand response. Conventional Model-Free (MF) and Model-Based (MB) RL algorithms, however, use a fixed reward model to learn the value function rendering their application challenging for ever changing SC environments. Successor Representations (SR)-based techniques are attractive alternatives that address this issue by learning the expected discounted future state occupancy, referred to as the SR, and the immediate reward of each state. SR-based approaches are, however, mainly developed for single agent scenarios and have not yet been extended to multi-agent settings. The paper addresses this gap and proposes the Multi-Agent Adaptive Kalman Filtering-based Successor Representation (MAKF-SR) framework. The proposed framework can adapt quickly to the changes in a multi-agent environment faster than the MF methods and with a lower computational cost compared to MB algorithms. The proposed MAKF-SR is evaluated through a comprehensive set of experiments illustrating superior performance compared to its counterparts.

show abstract

“…In recent years, autonomous underwater vehicles (AUVs), have undertaken important roles in oceanographic research and geoscience studies [1,2], plume tracing [3], oil spills and pipeline monitoring [4,5], underwater archaeological surveying [6], mapping flow fields [7], inspection [8], sampling [9] and tracking marine life [10] etc. In light of these autonomous applications, in this paper a solution to the issue of trajectory tracking is proposed for an under-actuated underwater vehicle.…”

Section: Introductionmentioning

confidence: 99%