This work investigates the stability of Halanay inequality. Some novel results are obtained by means of constructing an auxiliary differential equation. Some previous works are improved and extended. After that, the obtained results are applied to investigate the stability of neural networks with time-varying and distributed delays. At last, some examples along with numerical simulations are presented to illustrate the validity of the theoretical results. INDEX TERMS Stability, Halanay inequality, delay neural networks.