No abstract

No abstract

Unmanned Aerial Vehicles (UAVs) are abundantly becoming a part of society, which is a trend that is expected to grow even further. The quadrotor is one of the drone technologies that is applicable in many sectors and in both military and civilian activities, with some applications requiring autonomous flight. However, stability, path planning, and control remain significant challenges in autonomous quadrotor flights. Traditional control algorithms, such as proportional-integral-derivative (PID), have deficiencies, especially in tuning. Recently, machine learning has received great attention in flying UAVs to desired positions autonomously. In this work, we configure the quadrotor to fly autonomously by using agents (the machine learning schemes being used to fly the quadrotor autonomously) to learn about the virtual physical environment. The quadrotor will fly from an initial to a desired position. When the agent brings the quadrotor closer to the desired position, it is rewarded; otherwise, it is punished. Two reinforcement learning models, Q-learning and SARSA, and a deep learning deep Q-network network are used as agents. The simulation is conducted by integrating the robot operating system (ROS) and Gazebo, which allowed for the implementation of the learning algorithms and the physical environment, respectively. The result has shown that the Deep Q-network network with Adadelta optimizer is the best setting to fly the quadrotor from the initial to desired position.

Industrial control systems (ICSs) are more vulnerable to cyber threats owing to their network connectivity. The intrusion detection system (IDS) has been deployed to detect sophisticated cyber-attack but the existing IDS uses the packet header information for traffic flow detection. IDS is inefficient to detect packet deformation; therefore, we propose the adoption of packet payload in IDS to respond to a variety of attacks and high performance. Our proposed model detects packet modification and traffic flow by inspecting each packet and sequence of packets. For evaluation, cross verification is conducted to increase the reliability of the statistics.

Device-to-device communications are considered as a key feature to enhance the performance of the fifth generation (5G) wireless networks. Several radio access technologies such as LTE Direct, Bluetooth, Wi-Fi, and ZigBee are expected to provide the opportunity of D2D communications. Therefore, it is possible to choose any of them autonomously to establish a D2D link. The primary focus of this work is to investigate the radio interface selection, where end users select an interface opportunistically among different available radio interfaces to establish outband D2D connectivity against interference. We model a non-cooperative game to select a radio interface for D2D users to minimize their communication cost. We have investigated Nash equilibrium in the game and argue that without any cooperation users can achieve a balanced strategy. In our model, each pair selects a radio interface based on a utility function that associates communication performance and cost. Finally, we propose three heuristic algorithms: Social, Greedy, and Local, that achieve Nash equilibrium with different information. Event-driven simulation experiments are then conducted to evaluate the utility and cost of the equilibrium strategy. Our results confirm that the proposed schemes can increase the utility, lower the cost, and lead to higher efficiency in terms of achievable throughput per consumed energy. INDEX TERMS D2D communications, game theory, multiple radio interfaces, Nash equilibrium. I. INTRODUCTION Wireless communications have boosted the opportunity for smart devices with a number of standards and technologies. Smart devices are now the most important computing and communication platform. In previous years wireless connectivity was only possible with a single operator/ technology. However, these days smart devices are capable of multiple wireless opportunities. These end users are often equipped with multiple radio interfaces (i.e., 3G/LTE, Bluetooth, Zigbee, and Wi-Fi), which complements their cellular communication capabilities. According to a recent market research report, 70% of the mobile phones have Bluetooth interface, while 80% are enabled WiFi [1]. The proliferation of smart devices and exponential demand of bandwidth have created spacious performance requirement on the future wireless networks [2]. Device-to-device communication (D2D) [3] is considered one of the major technology to enhance the boosting demand [4] of users. Motivated by the performance gain, many telecommunication The associate editor coordinating the review of this manuscript and approving it for publication was Fang Yang.