SummaryWe study the energy harvesting (EH)‐assisted system model based on the performance of a dual‐hop cooperative communication system that is subjected to Nakagami‐ fading. Through the partial relay selection method, the selection of th best relay (BR) is performed among amplify and forward (AF) relays, which can harvest energy from radio frequency signals. At the receiver, the selection combining scheme is considered to select between the signals of th best relaying path and the direct path. For this considered system, we compute the closed‐form expressions of outage probability (OP) and the average symbol error rate (ASER) for higher order quadrature amplitude modulation (QAM) techniques, especially for rectangular QAM, cross QAM, and hexagonal QAM. Further, a new moment‐generating function expression is obtained which is used to derive the ASER expression related to the generalized non‐coherent modulation technique. We also give the asymptotic expression of OP to find out the diversity order. Furthermore, we study the effect of fading parameters, th BR, and other factors on system behavior. Finally, we verify the derived expressions with Monte Carlo simulations.