Soil organic matter (SOM) is one of the most important indicators of soil quality. Hyperspectral remote sensing technology has been recognized as an effective method to rapidly estimate SOM content. In this study, 173 samples (0–20 cm) were collected from farmland soils in the northwestern arid zones of China. Partial least squares regression (PLSR), support vector machine regression (SVMR), and random forests regression (RFR), based on 15 types of mathematical transformations of the original spectral data of soil, were applied for identifying the optimal estimation method. Distribution of SOM content was mapped using both ground-measured values and predicted values estimated based on the optimum models. Obtained results indicated that the important spectral wavebands with the highest correlation were identified as 421 nm, 441 nm, 1014 nm, 1045 nm, and 2351 nm for SOM in the soil. Spectral transformations had obvious effects on the spectral characteristics of SOM. The optimal estimation was obtained when RFR was combined with the reciprocal logarithmic first-order differential (RLFD) (R2 = 0.884, RMSE = 2.817%, MAE = 2.222) for SOM contents. Finally, the RFR-RLFD method had much better performance compared with the PLSR and SVMR models. Results of this study can provide an alternative to the application of the hyperspectral estimation of SOM in farmland soils in arid zones.