Apples are usually bagged during the growing process, which can effectively improve the quality. Establishing an in situ nondestructive testing model for in-tree apples is very important for fruit companies in selecting raw apple materials for valuation. Low-maturity apples and high-maturity apples were acquired separately by a handheld tester for the internal quality assessment of apples developed by our group, and the effects of the two maturity levels on the soluble solids content (SSC) detection of apples were compared. Four feature selection algorithms, like ant colony optimization (ACO), were used to reduce the spectral complexity and improve the apple SSC detection accuracy. The comparison showed that the diffuse reflectance spectra of high-maturity apples better reflected the internal SSC information of the apples. The diffuse reflectance spectra of the high-maturity apples combined with the ACO algorithm achieved the best results for SSC prediction, with a prediction correlation coefficient (Rp) of 0.88, a root mean square error of prediction (RMSEP) of 0.5678 °Brix, and a residual prediction deviation (RPD) value of 2.466. Additionally, the fruit maturity was predicted using PLS-LDA based on color data, achieveing accuracies of 99.03% and 99.35% for low- and high-maturity fruits, respectively. These results suggest that in-tree apple in situ detection has great potential to enable improved robustness and accuracy in modeling apple quality.