Along with the recent advance in wireless networking and data processing technologies, demands for low latency communication (LLC) are increasing in a wide variety of future-driven autonomous applications such as a smart factory, self-driving cars, and so on. The fifth generation of cellular mobile communications (5G) will cover this need as one of three key capacities in their usage scenarios: enhanced mobile broadband (eMBB), massive machine type communication (mMTC), and ultra-reliable low-latency communications (URLLC). The 5G systems are composed of mobile devices and various internet of things (IoT) devices for sensing, acting, and information services; they configure diverse networking topologies such as direct mobile-to-mobile, also known as device-to-device (D2D). In the 5G D2D network systems, the network topologies are easily broken because of the mobile devices such as smartphones, IoT devices, and so on. Thus, for the highly flexible and extensible 5G D2D network systems, mobility support for the mobile devices is necessary. In this paper, we first explore the mobility issues in beyond 5G D2D. Since there are static and mobile elements in the 5G application domains such as the smart factory, overall mobility would lead to highly frequent topology reconfiguration or connectivity reconstruction. Thus, latency-related problems derived from topology changes and connectivity failures due to the mobility are addressed. To handle the problems, a fast connectivity construction scheme, denoted by LMK, is proposed with a deep neural network dealing with learning on radio signal information in order to achieve the LLC. Evaluation results demonstrate that the proposed framework can provide reliable connectivity for the MAC layer link with a low latency data transmission.