Automatic recognition of human emotion has become an interesting topic among braincomputer interface (BCI) researchers. Emotion is one of the most fundamental features of a human subject. With proper analysis of emotion, the inner state of a human subject can be assessed directly. The human brain response can be competently represented by electroencephalography (EEG). The selection of potential features in EEG related to human emotion is a very important task for developing an effective emotion recognition system. In this paper, the discriminative features computed from rhythmic components of EEG are used to recognize human emotional states. The narrowband rhythmic components theta, alpha, beta, and gamma are extracted from multichannel EEG signals using filter bank implementation. The short-time entropy and energy features are extracted from each of the rhythmic components. The spatial filtering has been performed on the entropy-energy space by using common spatial pattern (CSP). Thus obtained spatial features are employed to recognize the emotion states using support vector machine (SVM) classifier. The publicly available two datasets DEAP and SEED are used to evaluate the performance of the proposed method. The experimental results reflect that higher recognition accuracy is obtained by using higher frequency subbands (beta and gamma) than that of the lower frequency subbands (theta and alpha). The combination of features from all subbands has better performance than the features obtained from individual subband signals. The performance of the proposed method outperforms the recently developed algorithms of emotion recognition.INDEX TERMS Brain-computer interface (BCI), common spatial pattern (CSP), electroencephalography (EEG), emotion recognition, subband decomposition.