Due to lack of robustness, Nonlinear Dynamic Inversion (NDI) control cannot resist the influence of disturbances. In this paper, an adaptive fault‐tolerant control method, called Improved Model Reference‐based Adaptive NDI (IMR‐ANDI), is proposed and applied to develop a fault‐tolerant flight controller to improve robustness to faults and disturbances. To improve the estimation accuracy, this paper also modified the piecewise‐constant adaptation, which improves the accuracy of adaptive estimation without increasing the sampling frequency. Moreover, a low‐pass filter is introduced to the control law creatively to weaken the disturbance of high‐frequency dynamics to the adaptation, which balances the dynamic performance and robustness as well as prevents the system oscillation. Finally, a fault‐tolerant flight controller is developed based on the IMR‐ANDI and verified under the actuator fault and center of gravity variations. All simulation and hardware‐in‐the‐loop experiment results conform to the theoretical analysis, indicating that the developed IMR‐ANDI fault‐tolerant controller is robust to faults and disturbances and can achieve expected control performance.