The mode switching between spatial multiplexing (SM) and space-time block code (STBC) diversity is investigated for the multiple-input multiple-output (MIMO) automatic repeat request (ARQ) system. Five important practical factors are considered in the proposed switching scheme: transmit correlation, ARQ technique, packet loss probability (PLP) constraint, discrete rate transmission (DRT) and channel coding. Under the spatially correlated channel, the distributions of the post signal-to-interference-plusnoise ratio (SINR) for the SM mode and the STBC mode are obtained by using Gamma approximations. Then this paper derives the closed-form expressions of the PLP and the throughput for different modes when the ARQ technique is employed, based on which the mode switching algorithm is proposed to improve the spectral efficency. In the simulation, the correction of the expressions is first verified. Then, the significant gain observed by the proposed algorithm is presented. Since the switching point is the key parameter to implement the mode switching, this paper also shows how the switching point is affected by the practical factors considered.