The next frontier in communications is teleoperation -manipulation and control of remote environments with feedback. Compared to conventional networked applications, teleoperation poses widely different requirements, ultra-low latency (ULL) is primary. Realizing ULL communication demands significant redesign of conventional networking techniques, and the network infrastructure envisioned for achieving this is termed as Tactile Internet (TI). The design of the network infrastructure and meaningful performance metrics are crucial for seamless TI communication. However, existing performance metrics fall severely short of comprehensively characterizing TI performance. We take the first step towards bridging this gap. We take Dynamic Time Warping(DTW) as the basis of our work and identify necessary changes for characterizing TI performance. Through substantial refinements to DTW, we design Effective Time-and Value-Offset (ETVO) -a new method for measuring the fine-grained performance of TI systems. Through an in-depth objective analysis, we demonstrate the improvements of ETVO over DTW. Through human-in-the-loop subjective experiments, we demonstrate how and why existing QoS and QoE methods fall short of estimating the TI session performance accurately. Using subjective experiments, we demonstrate the behavior of the proposed metrics, their ability to match theoretically derived performance, and finally their ability to reflect user satisfaction in a practical setting. The results are highly encouraging.

<div>Deep Reinforcement Learning (DRL) has the potential to surpass human-level control in sequential decision-making problems. Evolution Strategies (ESs) have different characteristics than DRL, yet they are promoted as a scalable alternative. </div><div>To get insights into their strengths and weaknesses, in this paper, we put the two approaches side by side. After presenting the fundamental concepts and algorithms for each of the two approaches, they are compared from the perspectives of scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the paper discusses hybrid algorithms, combining aspects of both DRL and ESs, and how they attempt to capitalize on the benefits of both techniques. Lastly, both approaches are compared based on the set of applications they support, showing their potential for tackling real-world problems.</div><div>This paper aims to present an overview of how DRL and ESs can be used, either independently or in unison, to solve specific learning tasks. It is intended to guide researchers to select which method suits them best and provides a bird's eye view of the overall literature in the field. Further, we also provide application scenarios and open challenges. </div>

No abstract

No abstract

No abstract