SUMMARY
This article presents a new approach for power transformer protection based on a hybrid pattern recognition scheme. The hyperbolic S‐transform (HST) is a very powerful tool for analysis of the nonstationary waveforms because it is able to extract the information from transient signals simultaneously in both time and frequency domains. Magnitude, frequency, and HST contours are the main attributes obtained from the output matrix of HST. At first, differential current waveforms of different conditions such as normal, internal and external faults, inrush, and over‐excitation conditions are analyzed by HST and some potential useful features are extracted from the abovementioned contours. To decrease the dimension of feature vector and to increase the classification accuracy of the proposed algorithm, the most effective features are selected by using well‐known feature selection methods namely sequential forward selection, sequential backward selection, and genetic algorithm. Selected features are trained by a probabilistic neural network as an effective classifier core, which has advantages regarding learning speed and generalization capability compared with feed‐forward neural network. The classification accuracy of the proposed algorithm has been used as a criterion function for the selection of the best subset features. The proposed protection scheme is evaluated for various operating conditions of three different transformers using the PSCAD/EMTDC package. Extensive simulation results show that the proposed algorithm relies only on the waveshape properties, and it is independent of the value of transformer parameters and consumed power. Copyright © 2012 John Wiley & Sons, Ltd.