Beamforming and Power Control in Sensor Arrays Using Reinforcement Learning

Almeida, Náthalee C.; Fernandes, Marcelo A. C.; Neto, Adrião Duarte Dória

doi:10.3390/s150306668

Cited by 7 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the Q-learning convergence criterion requires state-action pairs to be visited infinite times, in practice it is possible to reach quite relevant values when executing a sufficiently large number of iterations (considering the task to be learned). For a problem of 18 states and 2 actions, as described in [2], where Q-learning was used to determine an optimum selection policy between beam conformation and power control of an adaptive arrangement of antennas, the matrix Q convergence happens after approximately 2500 iterations. In the work of [29], Q-learning was used to make adaptive thermal management of multicores systems, in order to improve the reliability and extend their useful life.…”

Section: Q-learning Techniquementioning

confidence: 99%

“…In the problem presented in [29], which has the same number of states and actions of scenario III as the value function matrix, which converges with 5500 iterations, it would need 0.2 ms. In the problem presented in [2], with 2500 iterations, it would take 0.1 ms. If there is a time restriction to acquire the necessary data for the calculation of the policy, it is also possible to reduce the system clock so that, as demonstrated in [11], the system consumption is reduced and the execution time is adjusted for the data acquisition.…”

Section: E Power Consumption Analysismentioning

confidence: 99%

“…Reinforcement learning (RL), is an artificial intelligence formalism that allows an agent to learn from the interaction with the environment where it is inserted [1]. This approach is indicated for situations in which there is not enough information about the behavior that the agent must take to reach its objective, that is, the agent without previous knowledge learns through interaction with the environment, receiving rewards for his actions and finding, the optimal policy [2].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA

2019

View full text Add to dashboard Cite

Q-learning is an off-policy reinforcement learning technique, which has the main advantage of obtaining an optimal policy interacting with an unknown model environment. This paper proposes a parallel fixed-point Q-learning algorithm architecture implemented on field programmable gate arrays (FPGA) focusing on optimizing the system processing time. The convergence results are presented, and the processing time and occupied area were analyzed for different states and actions sizes scenarios and various fixed-point formats. The studies concerning the accuracy of the Q-learning technique response and resolution error associated with a decrease in the number of bits were also carried out for hardware implementation. The architecture implementation details were featured. The entire project was developed using the system generator platform (Xilinx), with a Virtex-6 xc6vcx240t-1ff1156 as the target FPGA.

show abstract

Section: Q-learning Techniquementioning

confidence: 99%

Section: E Power Consumption Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA

2019

View full text Add to dashboard Cite

show abstract

“…A power control schemes based on the reinforcement learning (RL) and mathematical optimization are proposed to consider interference. To determine the optimal combination of beamforming and power control in sensor arrays, the RL algorithm is proposed [ 24 ]. In this case, the power configuration set of this kind of scheme could be explored by [ 25 ].…”

Section: Related Workmentioning

confidence: 99%

Bio-Inspired Distributed Transmission Power Control Considering QoS Fairness in Wireless Body Area Sensor Networks

Lee

Jung

Lee

2017

Sensors

View full text Add to dashboard Cite

Recently, the development of wireless body area sensor network (WBASN) has accelerated due to the rapid development of wireless technology. In the WBASN environment, many WBASNs coexist where communication ranges overlap with each other, resulting in the possibility of interference. Although nodes in a WBASN typically operate at a low power level, to avoid adversely affecting the human body, high transmission rates may be required to support some applications. In addition to this, since many varieties of applications exist in the WBASN environment, each prospective user may have different quality of service (QoS) requirements. Hence, the following issues should be considered in the WBASN environment: (1) interference between adjacent WBASNs, which influences the performance of a specific system, and (2) the degree of satisfaction on the QoS of each user, i.e., the required QoS such as user throughput should be considered to ensure that all users in the network are provided with a fair QoS satisfaction. Thus, in this paper, we propose a transmission power adjustment algorithm that addresses interference problems and guarantees QoS fairness between users. First, we use a new utility function to measure the degree of the satisfaction on the QoS for each user. Then, the transmission power of each sensor node is calculated using the Cucker–Smale model, and the QoS satisfaction of each user is synchronized dispersively. The results of simulations show that the proposed algorithm performs better than existing algorithms, with respect to QoS fairness and energy efficiency.

show abstract

“…In this paper, we further optimize COPMA and RMSG techniques to minimize the overhead and power consumption respectively. In [26] the reinforcement learning (RL) based algorithm used to determine the suitable policy for selection between two different techniques, first one is beamforming while the second is power control for sensor array networks, by taking the benefit of the individual features of each method according to SINR threshold. The key issue of the RL is to select the action (BF or PC) according to the learning state in this case (SINR), that is satisfied with the system requirement.…”

Section: Introductionmentioning

confidence: 99%

Optimized Intelligent Design for Smart Systems Hybrid Beamforming and Power Adaptation Algorithms for Sensor Networks Decision-Making Approach

Khiarullah¹,

Tureli²,

Kivanc³

2019

IJCIS

View full text Add to dashboard Cite

During last two decades, power adaptation and beamforming solutions have been proposed for multiple input multiple output (MIMO) Ad Hoc networks. Game theory based methods such as cooperative and non-cooperative joint beamforming and power control for the MIMO ad hoc systems consider the interference and overhead reduction, but have failed to achieve the tradeoff between communication overhead and power minimization. Cooperative method using game theory achieves the power minimization, but introduced the overhead. The non-cooperative solution using game theory reduced the overhead, but it takes more power and iterations for convergence. In this paper, a novel game theory based algorithms proposed to achieve the tradeoff between power control and communication overhead for multiple antennas enabled wireless ad-hoc networks operating in multiple-users interference environment. The optimized joint iterative power adaption and beamforming method designed to minimize the mutual interference at every wireless node with constant received signal to interference noise ratio (SINR) at every receiver node. First cooperative potential game theory based algorithm designed for the power and interference minimization in which users cluster and binary weight books along used to reduce the overhead. Then the non-cooperative based approach using the reinforcement learning (RL) method is proposed to reduce the number of iterations and power consumption in networks, the proposed RL procedure is fully distributed as every transmit node require only an observation of its instantaneous beamformer label which can be obtained from its receive node. The simulation results of both methods prove the efficient power adaption and beamforming for small and large networks with minimum overhead and interference compared to state-of-art methods.

show abstract

Beamforming and Power Control in Sensor Arrays Using Reinforcement Learning

Cited by 7 publications

References 30 publications

Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA

Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA

Bio-Inspired Distributed Transmission Power Control Considering QoS Fairness in Wireless Body Area Sensor Networks

Optimized Intelligent Design for Smart Systems Hybrid Beamforming and Power Adaptation Algorithms for Sensor Networks Decision-Making Approach

Contact Info

Product

Resources

About