Out object in this paper it to study, the effect of nonlinearity on the bit error rate (BER) of MIMO systems in M-QAM modulation techniques. We consider Saleh’s model (power amplifier model) for the nonlinearity, and apply the nonlinear model on MIMO system with receiver diversity and transmitter diversity. For transmitter diversity, the Space-Time Block Coding (STBC) based on Alamouti scheme is used to provide transmits diversity for two transmitting antennas. The results show that, if there is a high variation in the amplitude of the M- QAM symbols, there will behigh effect of nonlinearity that causes high BER especially for high amplitude symbols at high SNR.