This paper is presented at 28th International Joint Conference on Artificial Intelligence (IJCAI-19) 1st Wksp. Federated Machine Learning for User Privacy and Data Confidentiality (FML'19), Macau, August 2019.

Traditional distributed deep reinforcement learning (RL) commonly relies on exchanging the experience replay memory (RM) of each agent. Since the RM contains all state observations and action policy history, it may incur huge communication overhead while violating the privacy of each agent. Alternatively, this article presents a communication-efficient and privacy-preserving distributed RL framework, coined federated reinforcement distillation (FRD). In FRD, each agent exchanges its proxy experience replay memory (ProxRM), in which policies are locally averaged with respect to proxy states clustering actual states. To provide FRD design insights, we present ablation studies on the impact of ProxRM structures, neural network architectures, and communication intervals. Furthermore, we propose an improved version of FRD, coined mixup augmented FRD (MixFRD), in which ProxRM is interpolated using the mixup data augmentation algorithm. Simulations validate the effectiveness of MixFRD in reducing the variance of mission completion time and communication cost, compared to the benchmark schemes, vanilla FRD, federated reinforcement learning (FRL), and policy distillation (PD).

Cognitive radio (CR) is a key enabler realizing future networks to achieve higher spectral efficiency by allowing spectrum sharing between different wireless networks. It is important to explore whether spectrum access opportunities are available, but conventional CR based on transmitter (TX) sensing cannot be used to this end because the paired receiver (RX) may experience different levels of interference, according to the extent of their separation, blockages, and beam directions. To address this problem, this paper proposes a novel form of medium access control (MAC) termed sense-andpredict (SaP), whereby each secondary TX predicts the interference level at the RX based on the sensed interference at the TX; this can be quantified in terms of a spatial interference correlation between the two locations. Using stochastic geometry, the spatial interference correlation can be expressed in the form of a conditional coverage probability, such that the signal-to-interference ratio (SIR) at the RX is no less than a predetermined threshold given the sensed interference at the TX, defined as an opportunistic probability (OP). The secondary TX randomly accesses the spectrum depending on OP. We optimize the SaP framework to maximize the area spectral efficiencies (ASEs) of secondary networks while guaranteeing the service quality of the primary networks. Testbed experiments using USRP and MATLAB simulations show that SaP affords higher ASEs compared with CR without prediction.

Opportunity detection at secondary transmitters (TXs) is a key technique enabling cognitive radio (CR) networks. Such detection however cannot guarantee reliable communication at secondary receivers (RXs), especially when their association distance is long. To cope with the issue, this paper proposes a novel MAC called sense-and-predict (SaP), where each secondary TX decides whether to access or not based on the prediction of the interference level at RX. Firstly, we provide the spatial interference correlation in a probabilistic form using stochastic geometry, and utilize it to maximize the area spectral efficiency (ASE) for secondary networks while guaranteeing the service quality of primary networks. Through simulations and testbed experiments using USRP, SaP is shown to always achieve ASE improvement compared with the conventional TX based sensing.

With the envisioned massive connectivity era, one of the challenges for 5G/Beyond 5G (B5G) wireless systems will be handling the unprecedented spectrum crunch. A potential solution has emerged in the form of spectrum sharing, which deviates from a monopolistic spectrum usage system. This paper investigates the medium access control (MAC) as a means of increasing the viability of the spectrum sharing technique. We first quantify the opportunity of spectrum access in a probabilistic manner, a method referred to as opportunistic (OP) map. Based on the OP framework, we propose a random MAC algorithm in which the access of a node is randomly determined with its own OP value. As a possible application of our OPmap based random MAC, we propose a flexible half-duplex (HD)/full-duplex (FD) communication where each pair decides the duplexing mode according to the OP values of the two pair nodes. This approach fits well with the spectrum sharing system since it enables a flexible operation for the spectrum access according to the spectrum usage level. From the numerical analysis, we validate the feasibility and verify the performance enhancements by implementing a field-programmable gate array (FPGA) based real-time prototype. We further carry out extensive 3D ray-tracing based system-level simulations on investigating the network-level performance of the proposed system. Measurements and numerical results confirm that the proposed architecture can achieve higher system throughput than conventional LTE-TDD (time division duplex) systems.INDEX TERMS Duplexing, dynamic spectrum access, dynamic spectrum management, cognitive radio, spectrum sharing, full-duplex radio, opportunistic spectrum access, wireless communication and spectrum sensing. II. SYSTEM ARCHITECTURE AND OP DETECTION WITH MAC SCHEMEThis section briefly introduces the proposed system architecture and concepts of our OP map with MAC analysis. Consider a general spectrum sharing system composed of communication nodes, spectrum sensors, and a distributed server that only provides the OP value (see Fig. 1). Deployed spectrum sensors periodically measure the interference level at their locations to check the spectrum usage level. Sensors send the measured results to the distributed server.