The oil‐immersed transformer is a crucial power equipment in power system, which is prone to abnormality or failure under long running. In order to improve the level of operation and maintenance and ensure the safe and reliable operation of the power grid, it is necessary to diagnose the fault of the oil‐immersed transformer in time. Most of the traditional fault vibration methods are off‐line detection, and are easily disturbed by mechanical deterioration. In order to solve the above problems, this paper established a 3D electromagnetic and stress coupling simulation model of oil‐immersed transformer by finite element simulation software. The vibration characteristics of oil‐immersed transformer core winding are obtained, and at the same time, this paper obtained the vibration signals at different positions on transformer core winding and oil tank. According to the vibration signal of the transformer, the best measuring point position of the tank wall is proposed to accurately reflect the vibration characteristics of the transformer. The signal data is preprocessed by CEEMDAN decomposition method, and the signal data is classified by GWO‐BP composite classification algorithm. This paper proposed an online transformer fault diagnosis method based on vibration signal, which achieves the purpose of accurately diagnosing oil‐immersed transformer faults. The test results show that in the case of oil‐immersed transformers the accuracy of the proposed transformer fault diagnosis method is more than 93%, which provides important guiding significance for the safe and stable operation of the transformer. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.