No abstract

No abstract

In reinforcement learning (RL), a reinforcement signal may be infrequent and delayed, not appearing immediately after the action that triggered the reward. To trace back what sequence of actions contributes to delayed rewards, e.g., credit assignment (CA), is one of the biggest challenges in RL. This challenge is aggravated under sparse binary rewards, especially when rewards are given only after successful completion of the task. To this end, a novel method consisting of key-action detection, among a sequence of actions to perform a task under sparse binary rewards, and CA strategy is proposed. The key-action defined as the most important action contributing to the reward is detected by a deep neural network that predicts future rewards based on the environment information. The rewards are re-assigned to the key-action and its adjacent actions, defined as adjacent-key-actions. Such re-assignment process enables increased success rate and convergence speed during training. For efficient re-assignment, two CA strategies are considered as part of proposed method. Proposed method is combined with hindsight experience replay (HER) for experiments in the OpenAI gym suite robotics environment. In the experiments, it is demonstrated that proposed method can detect key-actions and outperform the HER, increasing success rate and convergence speed, in the Fetch slide task, a type of task that is more exacting as compared to other tasks, but is addressed by few publications in the literature. From the experiments, a guideline for selecting CA strategy according to goal location is provided through goal distribution analysis with dot map.INDEX TERMS Credit assignment, delayed rewards, goal distribution, reinforcement learning, reward shaping.

Provision of energy to wireless sensor networks is crucial for their sustainable operation. Sensor nodes are typically equipped with batteries as their operating energy sources. However, when the sensor nodes are sited in almost inaccessible locations, replacing their batteries incurs high maintenance cost. Under such conditions, wireless charging of sensor nodes by a mobile charger with an antenna can be an efficient solution. When charging distributed sensor nodes, a directional antenna, rather than an omnidirectional antenna, is more energy-efficient because of smaller proportion of off-target radiation. In addition, for densely distributed sensor nodes, it can be more effective for some undercharged sensor nodes to harvest energy from neighboring overcharged sensor nodes than from the remote mobile charger, because this reduces the pathloss of charging signal due to smaller distances. In this paper, we propose a hybrid charging scheme that combines charging by a mobile charger with a directional antenna, and energy trading, e.g., transferring and harvesting, between neighboring sensor nodes. The proposed scheme is compared with other charging scheme. Simulations demonstrate that the hybrid charging scheme with a directional antenna achieves a significant reduction in the total charging time required for all sensor nodes to reach a target energy level.

No abstract