In this paper, a new methodology for the identification and equalization of massive Multiple-Input Multiple-Output (MIMO) channels in fifth generation (5G) wireless communications is proposed. Channel equalization is performed in state-space, having estimated the channel using the fractional-order Multivariable Output Error State Space (MOESP) system identification algorithm. The proposed fractional-order algorithm is an improvement over its integer-order counterpart that is currently used for state-space channel identification/ estimation and state-space channel equalization. When dealing with fractional-order calculus our work adopts the Riemann-Liouville definition. Our numerical results of channel identification having used a chirp signal to excite our massive MIMO system show a lower mean square error (MSE) in channel estimation using the proposed fractional-order MOESP identification algorithm when compared to integer-order MOESP identification algorithm of increased order. Furthermore, following equalization, transmission using Binary phase shift keying (BPSK), Quadrature phase shift keying (QPSK) and 256-Quadrature amplitude modulation (256-QAM) signals show that the symbol error rate (SER) as a function of signal to noise ratio (SNR) performance of the fractional-order equalization algorithm compares to that of the integer-order equalization algorithm. The proposed channel identification algorithm also provides a more parsimonious solution for modelling multipath fading, thus enabling the design of fractional-order equalizers. In addition, they may be used in other applications where high order filtering and more complex control algorithms which are difficult to tune would be needed. Finally, the work has other technological applications where there is a requirement for modelling and control of propagation processes in dispersive media.