Solid wood flooring has good esthetic properties and is an excellent material for interior decoration. To meet the artistic effects of specific interior decoration requirements, the color of solid wood flooring needs to be coordinated. Thus, the color of the produced solid wood flooring needs to be sorted to meet the individual needs of customers. In this work, machine vision, deep learning methods, and ensemble learning methods are introduced to reduce the cost of manual sorting and improve production efficiency. The color CCD camera was used to collect 108 solid wood floors of three color grades provided by the company and obtained 108 18,000 × 2048 pixel wood images. A total of 432 images were obtained after data expansion. Deep learning methods, such as VGG16, DenseNet121, and XGBoost, were compared. After using XGBoost to filter the features, the accuracy of solid wood flooring color classification was 97.22%, the training model time was 5.27 s, the average test time for each picture was 51 ms, and a good result was achieved.