As the first visual element, color is the most attractive in the forest landscape. There are various kinds of forest colors; however, the human eye’s ability to recognize them is limited. In order to combine color composition and human eye recognition ability to quantify forest colors more appropriately and to improve the ornamental effect of forest color landscapes more precisely, we have constructed a forest color palette using k-means clustering based on the color information of 986 forest images from 40 national forest parks in China. The differences in color recognition accuracy and sensitivity among populations and colors were analyzed. The effect of forest color patch indices on color identification accuracy for interior and distant forest landscapes was also explored. The results were as follows: (1) forest color could be divided into eight color families—orange, yellow, yellow-green, green, blue-green, blue, purple, and red. (2) For humans, the recognition accuracy was highest for green and lowest for blue-green. (3) For interior forest landscapes, the mean area proportion and fractal dimension of the color patches showed significant positive effects on color recognition accuracy, whereas the number and density of color patches showed significant negative effects. For distant forest landscapes, the density and Shannon’s diversity index of the color patches showed significant positive effects for color recognition accuracy, whereas the number, edge density, division index, and cohesion of the color patches showed significant negative effects. We thus suggest that it is necessary to increase the complexity of the color patch shape when creating interior forest landscapes and to focus on the diversity and balance of color matching when creating distant forest landscapes. In future studies, the collection pathways for forest images should be expanded, and color information extraction algorithms that incorporate human perception should be selected. This will improve the data available for forest color studies and enable the construction of a more accurate forest color palette.