The core of harmonic detection is the recognition and extraction of each order harmonic in the signal. The current detection methods are seriously affected by the fence effect and spectrum aliasing, which brings great challenges to the detection of each order harmonic in the signal. This paper proposes an efficient harmonic detection neural network based on all-phase processing. It is based on three crucial designs. First, a harmonic signal-processing module is developed to ensure phase invariance and establish the foundation for subsequent modules. Then, we constructed the backbone network and utilized the feature-extraction module to extract deep abstract harmonic features of the target. Furthermore, a channel attention mechanism is also introduced in the weight-selection module to enhance the energy of the residual convolution stable spectrum feature, which facilitates the accurate and subtle expression of intrinsic characteristics of the target. We evaluate our method based on frequency, phase, and amplitude in two environments with and without noise. Experimental results demonstrate that the proposed EhdNet method can achieve 94% accuracy, which is higher than the compared methods. In comparison experiments with actual data, the RMSE of EhdNet is also lower than that of other recent methods. Moreover, the proposed method outperforms ResNet, BP, and other neural network approaches in data processing across diverse working conditions due to its incorporation of a channel attention mechanism.