Emotions carry the token indicating a human's mental state. Understanding the emotion exhibited becomes difficult for people suffering from autism and alexithymia. Assessment of emotions can also be beneficial in interactions involving a human and a machine. A system is developed to recognize the universally accepted emotions such as happy, anger, sad, disgust, fear and surprise. The gender of the speaker helps to obtain better clarity for identifying the emotion. Hidden Markov Model serves the purpose of gender identification.