In the next-generation wireless networks, cooperative communication is one of the auspicious techniques through which spatial diversity could be achieved by permitting the single antenna to act as virtual multiple input multiple output (VMIMO). The fundamental principle of cooperative communication was established on different types of relaying protocols and implementation of different relaying protocols according to the requirements of communication scenarios. The challenging task for achieving a high performance in cooperative communication is to find out the optimal relay node (RN) among different prevailing RNs. The basic purpose of cooperative communication is to maximize the performance of the network and minimize the overhead triggered by RNs by considering different communication metrics, i.e., signal to noise ratio (SNR), channel state information (CSI), and bit error rate (BER). This study presents a review of different cooperative relaying protocols with best relay selection techniques in the next-generation wireless network. Moreover, the different challenges faced by millimeter wave in 5G wireless networks and the role of cooperative communication to overcome those challenges are discussed. Turkey is that it transmits the duplicates of a signal to the destination independently. Many ideas regarding cooperative communication were first derived by Nichola et al. [5]. In the next-generation wireless network, cooperative communication may be one of the promising approaches for achieving the high data rates as well as efficient utilization of the bandwidth. Cooperative communication uses a RN, to provide coverage in the holes within the Long-Term Evolution-Advanced (LTE-A) cellular networks [6]. Similarly, in mm-wave communication that is considered in 5G, the relaying techniques are used to overcome different challenges of link blockage, backhaul connectivity, and path loss etc. Moreover, to make this technology more efficient and reliable, further improvements are required to achieve these goals.This study presents a broad overview of different relaying protocols with optimal relay selection methods used in different wireless networks. In addition, this study presents the potential benefits of using cooperative communication to overcome different challenges faced by the next generation of wireless networks. The remainder of this article is organized as follows. Section 2 presents existing research on cooperative communication. Section 3 gives an overview of cooperative communication, in which different relaying protocols are presented and comparison between them. In Section 4, we discuss the review of optimal relay selection techniques. In Section 5, we explain the different challenges faced by mm-wave and how to overcome them by cooperative communication. Section 6 summarizes the future challenges in cooperative communication and finally the conclusion is given in Section 7. Sami et al. [7] demonstrate a survey and taxonomy on medium access control for cooperative communication. This study classifies
...