The objective of this study is to classify the rice grains of seventeen different varieties popularly planted in Vietnam. Image processing is used to extract color, morphological, and texture features of the rice grains. Five feature subsets are formed, namely, morphological, basic color, clustering color, statistical, and gray level co-occurrence matrix (GLCM). These subsets and combined sets are evaluated for classification ability with a support vector machine (SVM). A dataset of 248 features, including a total of color, morphological, and texture features classified with the SVM gives an overall accuracy of 88.29%. To decrease the number of used features and to improve the classification accuracy, the proposed method combining binary particle swarm optimization (BPSO) and the SVM, called BPSO+SVM, is applied to the dataset. In the results, classification accuracy from BPSO+SVM reaches 93.94% using only 96 selected features. The obtained result shows the proposed method achieves higher classification accuracy than the SVM alone, and the required number of features is only 39% of the total dataset. This result can be applied for developing an automatic classification and identification system of rice varieties.INDEX TERMS Rice varieties, color features, morphological features, texture features, support vector machine, binary particle swarm optimization.