In this paper, we study the performance of the two-way relay network (TWRN) with NOMAbased transmission, where the information is exchanged during three time slots. To provide a fair comparison, we also examine the performance of a three-time-slot (3T) OMA-based transmission scheme. In the following, these schemes are referred to as 3T-NOMA-TWRN and 3T-OMA-TWRN. In the 3T-NOMA-TWRN scheme, each transceiver simultaneously transmits two symbols with different powers to exploit the multiplexing gain, implemented through the power-domain NOMA technique. The performance of both schemes is analyzed in terms of the outage probability, the ergodic sum-rate, and their asymptotic behaviors. The outage and rate performances of the schemes are enhanced by optimizing the power allocation coefficients, especially in the NOMA-based scheme. The proposed power allocation algorithm imposes negligible overhead because of considering the closed-form expressions of the outage probability and the ergodic sum-rate as the objective function. For the NOMA-based scheme, we propose sub-optimal power allocation strategies that closely approximate the optimal solution. The validity of the analytical results is confirmed by extensive numerical examples. The results show that the 3T-NOMA-TWRN scheme outperforms its OMAbased counterpart in terms of the outage probability and the ergodic sum-rate, especially in the high-rate and high-SNR regimes. Additionally, the 3T-NOMA-TWRN scheme significantly outperforms the existing twotime-slot (2T)-NOMA-TWRN scheme in the high-SNR regime. However, in the low-SNR regime, only in some cases, the situation changes in favor of the 2T-NOMA-TWRN scheme. Surprisingly, the performance of the 3T-OMA-TWRN scheme can significantly surpass that of the 2T-NOMA-TWRN scheme in certain situations, depending on the rate and SNR regimes and the network geometry. Furthermore, the 3T-NOMA-TWRN scheme significantly reduces the TWRN performance sensitivity to the location of relay node, which is a critical concern in both the 3T-OMA-TWRN and 2T-NOMA-TWRN schemes, especially in the latter case.INDEX TERMS Non-orthogonal multiple access (NOMA), power allocation, superposition coding, successive interference cancelation, two-way relay network (TWRN).
