The ability to recognize emotion is one of the hallmarks of emotion intelligence. This paper proposed to recognize emotion using physiological signals obtained from multiple subjects. IAPS (International Affective Picture System) images were used to elicit target emotions. Five physiological signals: Blood volume pulse (BVP), Electromyography (EMG), Skin Conductance (SC), Skin Temperature (SKT) and Respiration (RESP) were selected to extract 30 features for recognition. Two pattern classification methods, Fisher discriminant and SVM method are used and compared for emotional state classification. The experimental results indicate that the proposed method provides very stable and successful emotional classification performance as 92% over six emotional states.