Water-to-air unmanned aerial vehicle (UAV) based rolling shutter optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN)

Abstract

Underwater sensors and autonomous underwater vehicles (AUVs) are widely adopted in oceanic research activities. As the number of underwater sensors and AUVs is growing quickly, the bandwidth requirements are increasing accordingly. In this work, we put forward and demonstrate a large field-of-view (FOV) water-to-air unmanned aerial vehicle (UAV) based optical camera communication (OCC) system with gated recurrent unit neural network (GRU-NN) for the first time to the best of our knowledge. As the UAVs are embedded with complementary-metal-oxide-semiconductor (CMOS) cameras, there is no need to install OCC receivers (Rxs), reducing the deployment cost. Moreover, the large photo-sensitive area of the CMOS camera can support a large FOV OCC transmission without the need for precise optical alignment. Here, by utilizing the column matrix identification during the rolling shutter pattern decoding in the CMOS image sensor, the scintillation caused by water turbulence can be reduced. Besides, in the outdoor and windy environment, the UAV will experience significant movement caused by the wind making it very difficult to capture stable OCC frames in the CMOS image sensor. Here, we propose and demonstrate utilizing GRU-NN, which is a special realization of the recurrent neural network (RNN) with memory cells capable of learning the time-domain dependent signals. It is shown that the GRU-NN can learn effectively from successive image frames in time-domain and produce correct prediction even under the windy and unstable UAV flying environment. Experimental results reveal that the proposed GRU-NN can outperform the previous pixel-per-symbol labeling neural network (PPS-NN), and also can significantly reduce the computation time when compared with long-short-term-memory-neural-network (LSTM-NN). The proposed system can decode 4-level pulse-amplitude-modulation (PAM4) rolling shutter OCC patterns at data rates of 5.4 kbit/s and 3.0 kbit/s under clear and cloudy water, respectively, fulfilling the pre-forward error correction bit-error rate (pre-FEC BER = 3.8 × 10−3). We also demonstrate that the UAV based OCC system can support data rates of 5.4 kbit/s, 4.2 kbit/s, and 3.0 kbit/s at distances of 2.2 m, 3.2 m, and 4.2 m, respectively, at outdoor and windy environments.