To deduce the process of bruise and reduce the number of bruised fruits from the source, the storage time of yellow peaches after bruise should be identified. In order to distinguish the different storage times of mild bruise’s yellow peaches more effectively than current detection methods, the combined hyperspectral imaging and machine learning method was proposed. Firstly, the sample bruise region spectrum was extracted as spectral features, and then, the hyperspectral images were processed by Principal Component Analysis (PCA), and eight single-wavelength images were selected according to the weight coefficient curve of PC1 images, and the gray values of the selected images were calculated as image features. Finally, in order to find the optimal discriminative model, random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGBoost) models were built based on spectral features, image features, and spectral features combined with image features, respectively. The results show that the XGBoost models based on spectral features, image features, and spectral features combined with image features are the optimal models with the overall accuracy of 77.50%, 87.50% and 90.00%, respectively. To simplify the model, Competitive Adaptive Reweighted Sampling (CARS) algorithm was used to screen the wavelength of the normalized spectral data, and then they were fused with the image feature data again and the XGBoost model with an overall model accuracy of 95.00% was built. To sum up, the combined hyperspectral imaging and machine learning method can be used to distinguish the different storage times (2 h, 8 h, 24 h and 48 h) of mild bruise’s yellow peaches effectively. It provides a certain theoretical basis for hyperspectral imaging technology in fruit bruise detection.