This study introduced a detection method for magnesite and associated gangue minerals, including dolomite, calcite, and talc, based on mineralogical, chemical, and hyperspectral analyses using hand samples from thirteen different source locations and Specim hyperspectral short wave infrared (SWIR) hyperspectral images. Band ratio methods and logistic regression models were developed based on the spectral bands selected by the random forest algorithm. The mineralogical analysis revealed the heterogeneity of mineral composition for naturally occurring samples, showing various carbonate and silicate minerals as accessory minerals. The Mg and Ca composition of magnesite and dolomite varied significantly, inferring the mixture of minerals. The spectral characteristics of magnesite and associated gangue minerals showed major absorption features of the target minerals mixed with the absorption features of accessory carbonate minerals and talc affected by mineral composition. The spectral characteristics of magnesite and dolomite showed a systematic shift of the Mg-OH absorption features toward a shorter wavelength with an increased Mg content. The spectral bands identified by the random forest algorithm for detecting magnesite and gangue minerals were mainly associated with spectral features manifested by Mg-OH, CO3, and OH. A two-step band ratio classification method achieved an overall accuracy of 92% and 55.2%. The classification models developed by logistic regression models showed a significantly higher accuracy of 98~99.9% for training samples and 82–99.8% for validation samples. Because the samples were collected from heterogeneous sites all over the world, we believe that the results and the approach to band selection and logistic regression developed in this study can be generalized to other case studies of magnesite exploration.