Color classification of wooden boards is helpful to improve the appearance of wooden furniture that is spliced from multiple wooden boards. Due to the similarity of colors among wooden boards, manual color classification is inaccurate and unstable. Thus, supervised learning algorithms can hardly be used in this scenario. Moreover, wooden boards are long, and their images have a high resolution, which may lead to the growth of computational complexity. To overcome these challenges, in this paper, we propose a new mechanism for color classification of wooden boards based on machine vision. The image of the wooden board is preprocessed to subtract irrelevant colors, and the feature vector is extracted based on 3D color histogram to reduce the computational complexity. In the offline clustering, the feature vector sets are partitioned into different clusters through the K-means algorithm. Then, the clustering result can be used in the online classification to classify the new wood image. Furthermore, to process the abnormal images of wooden boards, we propose an improved algorithm with centroid improvement and image filtering. The experimental results verify the effectiveness of the proposed mechanism.