Abstract-Music genre classification is a vital component for the music information retrieval system. There are two important components to be considered for better genre classification, which are audio feature extraction and classifier. This paper incorporates two different kinds of features for genre classification, timbral texture and rhythmic content features. Timbral texture contains the Mel-frequency Cepstral Coefficient (MFCC) with other several spectral features. Before choosing a timbral feature we explore which feature contributes a less significant role on genre discrimination. This facilitates the reduction of feature dimension. For the timbral features up to the 4-th order central moments and the covariance components of mutual features are considered to improve the overall classification result. For the rhythmic content the features extracted from beat histogram are selected. In the paper Extreme Learning Machine (ELM) with bagging is used as the classifier for classifying the genres. Based on the proposed feature sets and classifier, experiments are performed with two well-known datasets: GTZAN and the ISMIR2004 databases with ten and six different music genres, respectively. The proposed method acquires better and competitive classification accuracy compared to the existing approaches for both data sets.