SummaryThe future generation of wireless communications systems uses the generalized frequency division multiplexing (GFDM) due to its viable candidate waveform to perform multiple user scheduling. It is a non‐orthogonal waveform susceptible to intercarrier and intersymbol interference (ISI); still, it offers flexible pulse shaping that enhances the efficiency of user scheduling. Here, to accomplish spatial diversity, the multiple‐input multiple‐output (MIMO) is incorporated with GFDM to enhance the performance; there is also an extra inter‐antenna interference problem that limits the model's performance. The issue concerning inter‐antenna interference can be resolved by adopting pilot‐based information transfer. The detection of a signal at the receiver based on the minimum mean square error has the issue of computational complexity while enhancing the estimation quality of a detector. Hence, a low‐complexity channel estimation technique is proposed in this research using the V‐degree polynomial‐based channel estimation technique, wherein the cubic order computation is reduced to square order. Also, the proposed adaptive pulse shaping technique, wherein the filter coefficient is optimized using the gazelle optimization algorithm (GOA) to provide optimal pulse shaping filter parameters by employing bit error rate (BER) as the objective function. The performance of a proposed V‐degree polynomial‐based channel estimation is analyzed based on various assessment measures like BER and MSE and acquired the minimal values of 5.75E‐05 and 2.07E‐05, respectively.