A survey on autonomous environmental monitoring approaches: towards unifying active sensing and reinforcement learning

Abstract

The environmental pollution caused by various sources has escalated the climate crisis making the need to establish reliable, intelligent, and persistent environmental monitoring solutions more crucial than ever. Mobile sensing systems are a popular platform due to their cost-effectiveness and adaptability. However, in practice, operation environments demand highly intelligent and robust systems that can cope with an environment’s changing dynamics. To achieve this reinforcement learning has become a popular tool as it facilitates the training of intelligent and robust sensing agents that can handle unknown and extreme conditions. In this paper, a framework that formulates active sensing as a reinforcement learning problem is proposed. This framework allows unification with multiple essential environmental monitoring tasks and algorithms such as coverage, patrolling, source seeking, exploration and search and rescue. The unified framework represents a step towards bridging the divide between theoretical advancements in reinforcement learning and real-world applications in environmental monitoring. A critical review of the literature in this field is carried out and it is found that despite the potential of reinforcement learning for environmental active sensing applications there is still a lack of practical implementation and most work remains in the simulation phase. It is also noted that despite the consensus that, multi-agent systems are crucial to fully realize the potential of active sensing there is a lack of research in this area.

Distributed safe formation tracking control of multiquadcopter systems using barrier Lyapunov function

“…A multiagent system consists of several interacting intelligent agents that can cooperate their movements, sensing, and computations to achieve a common goal. Multiquadcopter systems are ideal solutions for different challenges imposed by humans working in extreme environments ( Burrell et al, 2018 ); for example, using multiple quadcopters improves the performances, time and energy efficiencies, coverage areas, and redundancies of multiple robots performing the same task ( Mansfield and Montazeri, 2024 ).…”

“…It is assumed that each drone can fly and share information with the other robots in its neighborhood. Formation control is used in many applications relevant to environmental monitoring, such as coverage, patrolling, autonomous exploration, search and rescue, source seeking, and boundary tracking ( Mansfield and Montazeri, 2024 ). In Liu and Bucknall (2018) , the problem of formation control and cooperative motion planning of multiple unmanned vehicles was investigated and various approaches were reviewed; this work provides good insights into the challenges and techniques available for cooperative path planning and formation control.…”

“…Recently, reinforcement learning (RL) and deep reinforcement learning (DRL) techniques have been proven to be effective for decision-making and operation of cooperative robots in complex environments under time-varying and uncertain conditions. For example, Mansfield and Montazeri (2024) reviewed different multiagent RL (MARL) techniques used as advanced tools in the design of optimal cooperative trajectories for multirobot systems in environmental monitoring applications by optimizing not only the individual rewards of each of the robots but also their collective reward; although the focus of this was control and not desired trajectory design for quads operating in uncertain and complex environments, it was assumed that the target trajectory of each robot was designed and made available using the techniques of Liu and Bucknall (2018) or Mansfield and Montazeri (2024) . Further, as mentioned in Mansfield and Montazeri (2024) , the RL technique can be used to avoid interagent collisions and obstacles.…”