As the COVID-19 spread worldwide, countries around the world are actively taking measures to fight against the epidemic. To prevent the spread of it, a high sensitivity and efficient method for COVID-19 detection is necessary. By analyzing the COVID-19 chest X-ray images, a combination method of image regrouping and ResNet-SVM was proposed in this study. The lung region was segmented from the original chest X-ray images and divided into small pieces, and then the small pieces of lung region were regrouped into a regular image randomly. Furthermore the regrouped images were fed into the deep residual encoder block for feature extraction. Finally the extracted features were as input into support vector machine for recognition. The visual attention was introduced in the novel method, which paid more attention to the features of COVID-19 without the interference of shapes, rib and other related noises. The experimental results showed that the proposed method achieved 93% accuracy without large number of training data, outperformed the existing COVID-19 detection models.