In view of the fact that current attention-recognition studies are mostly single-level-based, this paper proposes a multi-level attention-recognition method based on feature selection. Four experimental scenarios are designed to induce high, medium, low, and non-externally directed attention states. A total of 10 features are extracted from 10 electroencephalogram (EEG) channels, respectively, including time-domain measurements, sample entropy, and frequency band energy ratios. Based on all extracted features, an 88.7% recognition accuracy is achieved when classifying the four different attention states using the support vector machine (SVM) classifier. Afterwards, the sequence-forward-selection method is employed to select the optimal feature subset with high discriminating power from the original feature set. Experimental results show that the classification accuracy can be improved to 94.1% using the filtered feature subsets. In addition, the average recognition accuracy based on single subject classification is improved from 90.03% to 92.00%. The promising results indicate the effectiveness of feature selection in improving the performance of multi-level attention-recognition tasks.