This article proposes a novel mutual impedance matrix model for compensating mutual coupling effects in adaptive array with application to adaptive nulling of interference signals. In the new method, extreme care has been taken into account for both self impedance and mutual impedance, relating to the mutual coupling effects. Numerical simulation results demonstrate the robustness and capability of this technique. By using the new method, it is found that both the accuracy of the positioning and depth of the nulls are significantly improved. Performance comparisons of the new methodology and several other previous techniques via a number of simulation are presented. V C 2013 Wiley Periodicals, Inc.Int J RF and Microwave CAE 24:30-38, 2014.