Leaf water content (LWC) is a vital indicator of crop growth and development. While visible and near-infrared (VIS–NIR) spectroscopy makes it possible to estimate crop leaf moisture, spectral preprocessing and multiband spectral indices have important significance in the quantitative analysis of LWC. In this work, the fractional order derivative (FOD) was used for leaf spectral processing, and multiband spectral indices were constructed based on the band-optimization algorithm. Eventually, an integrated index, namely, the multiband spectral index (MBSI) and moisture index (MI), is proposed to estimate the LWC in spring wheat around Fu-Kang City, Xinjiang, China. The MBSIs for LWC were calculated from two types of spectral data: raw reflectance (RR) and the spectrum based on FOD. The LWC was estimated by combining machine learning (K-nearest neighbor, KNN; support vector machine, SVM; and artificial neural network, ANN). The results showed that the fractional derivative pretreatment of spectral data enhances the implied information of the spectrum (the maximum correlation coefficient appeared using a 0.8-order differential) and increases the number of sensitive bands, especially in the near-infrared bands (700–1100 nm). The correlations between LWC and the two-band index (RVI1156, 1628 nm), three-band indices (3BI-3(766, 478, 1042 nm), 3BI-4(1129, 1175, 471 nm), 3BI-5(814, 929, 525 nm), 3BI-6(1156, 1214, 802 nm), 3BI-7(929, 851, 446 nm)) based on FOD were higher than that of moisture indices and single-band spectrum, with r of − 0.71**, 0.74**, 0.73**, − 0.72**, 0.75** and − 0.76** for the correlation. The prediction accuracy of the two-band spectral indices (DVI(698, 1274 nm) DVI(698, 1274 nm) DVI(698, 1274 nm)) was higher than that of the moisture spectral index, with R2 of 0.81 and R2 of 0.79 for the calibration and validation, respectively. Due to a large amount of spectral indices, the correlation coefficient method was used to select the characteristic spectral index from full three-band indices. Among twenty seven models, the FWBI-3BI− 0.8 order model performed the best predictive ability (with an R2 of 0.86, RMSE of 2.11%, and RPD of 2.65). These findings confirm that combining spectral index optimization with machine learning is a highly effective method for inverting the leaf water content in spring wheat.