Self-interference (SI) cancellation for full-duplex (FD) multiple input multiple output (MIMO) systems is challenging due to both hardware and signal processing complexity. In this paper, a beam-based adaptive filter structure with analog least mean square (ALMS) loops is proposed to significantly reduce the complexity of SI cancellation for FD MIMO systems. With this structure, the number of adaptive filters required for SI cancellation scales linearly with the number of transmit beams rather than quadratically with the number of antennas. Furthermore, to avoid additional transmit chains used to upconvert the beam signals to generate reference signals for the ALMS loops, a novel method is proposed to select the optimized reference signals from all transmitted signals. In addition, our stationary analysis shows that the proposed structure for FD MIMO systems outperforms the ALMS loop employed for an FD single input single output system. Simulations are conducted to confirm the theoretical analyses.