Powering UAV with Deep Q-Network for Air Quality Tracking

Abstract

Tracking the source of air pollution plumes and monitoring the air quality during emergency events in real-time is crucial to support decision-makers in making an appropriate evacuation plan. Internet of Things (IoT) based air quality tracking and monitoring platforms have used stationary sensors around the environment. However, fixed IoT sensors may not be enough to monitor the air quality in a vast area during emergency situations. Therefore, many applications consider utilizing Unmanned Aerial Vehicles (UAVs) to monitor the air pollution plumes environment. However, finding an unhealthy location in a vast area requires a long navigation time. For time efficiency, we employ deep reinforcement learning (Deep RL) to assist UAVs to find air pollution plumes in an equal-sized grid space. The proposed Deep Q-network (DQN) based UAV Pollution Tracking (DUPT) is utilized to guide the multi-navigation direction of the UAV to find the pollution plumes’ location in a vast area within a short duration of time. Indeed, we deployed a long short-term memory (LSTM) combined with Q-network to suggest a particular navigation pattern producing minimal time consumption. The proposed DUPT is evaluated and validated using an air pollution environment generated by a well-known Gaussian distribution and kriging interpolation. The evaluation and comparison results are carefully presented and analyzed. The experiment results show that our proposed DUPT solution can rapidly identify the unhealthy polluted area and spends around 28% of the total time of the existing solution.

Fine grained analysis method for unmanned aerial vehicle measurement based on laser-based light scattering particle sensing

“…Lutz B et al [11] developed a multihelicopter system to investigate the aerosol particle, black carbon, ozone, NOx (nitrogen oxides) and CO concentration vertical distributions as well as the meteorological parameters of temperature and humidity. Mohammed Alaelddin F. Y. et al [12] proposed UAV Pollution Tracking based on the Deep Q Network for guiding UAVs in multiple navigational directions in order to discover the location of a large area of pollution plume in a short period. Ozge Kucukkor et al [13] designed a customized four-rotor UAV with a metal oxide semiconductor (MOS) type carbon monoxide (CO) sensor and data collection module for detecting and measuring CO contamination in industrial locations and in urban regions.…”

Internet of Things (IoT)-based indoor plant care system

“…Environmental pollution is considered one of the main drawbacks to people's health, due to both its negative influence on the quality of the air individuals breathe and its mortality rate [20,23]. Air pollution is caused by human beings themselves due to certain advances (means of transport, industry, heating in homes, etc.)…”

“…To overcome the disadvantages of the IoTSs proposed for the care of potted plants that we have analysed, this paper presents an IoTS, called P4L (Plants for Life), which is able to keep plants healthy, especially indoors, by providing adequate watering and light when needed. In addition, the system can notify the inhabitants of the house or workers in the office (or company) of the presence of gases, such as carbon dioxide (CO 2 ), ammonium (NH 4 ), alcohol, and toluene (C 6 H 5 CH 3 ), which represent a poor air quality [20], and alert them to inadequate or even dangerous levels for people's health. In this way, the aim is to avoid the inconvenience caused by indoor pollution.…”

Driving Profile Optimization Using a Deep Q‐Network to Enhance Electric Vehicle Battery Life

“…Terapaptommakol et al [19] proposed a deep Q-network (DQN) method to develop an autonomous vehicle control system to achieve trajectory design and collision avoidance with regard to obstacles on the road in a virtual environment. Mohammed et al [20] employed deep reinforcement learning to assist unmanned aerial vehicles to find air pollution plumes in an equal-sized grid space. Zheng et al [21] proposed to model the dynamic scheduling of automatic guided vehicles as a Markov decision process (MDP) with mixed decision rules based on a DQN to generate the optimal policy.…”