Determination of the myristicin and alpha-pinene content of nutmeg is still constrained by the extended testing time in the laboratory, which is expensive and is carried out destructively. In addition, non-destructive testing using spectroscopy often faces problems in building models that only rely on algorithms that perform linearly, such as PCR and PLSR. Therefore, the present study studied Vis-NIR (381-1065 nm) as a fast, inexpensive, and non-destructive mechanism to determine the myristicin and alpha-pinene of nutmeg fruits from Aceh, Indonesia. Two algorithms commonly used in spectral data processing, partial least squares regression (PLSR) and machine learning represented by a support vector machine (SVM), were employed and compared to predict myristicin and alpha-pinene in nutmeg fruits. The chemical reference parameters (myristicin and alpha-pinene) were measured using gas chromatography mass spectrometry (GC-MS). Standard normal variate (SNV) and multiplicative scatter correction (MSC) preprocessing were involved as spectra enhancement before the prediction models outcome. The results show that the kernel of the radial basis function (RBF) kernel n-SVM algorithm is better than PLSR for myristicin prediction with gamma (g), c, and nu (n) of 0.1, 1.0, and 0.99, respectively. Also, the e-SVM algorithm by RBF kernel is better than PLSR for the prediction of alpha-pinene in nutmeg fruits with gamma (g), c, and epsilon (e) compositions of 0.01, 10, 0.1, respectively. The coefficient correlation of calibration (rc) and coefficient determination of prediction (Rp2), the root means square error of calibration (RMSEC) and prediction (RMSEP), and the ratio (RPD) for the prediction of myristicin were 0.992, 0.986, 0.941%, 1.325% and 8.348, respectively. The coefficient correlation of calibration (rc) and coefficient determination of prediction (Rp2), the root mean square error of calibration (RMSEC) and prediction (RMSEP), and the ratio of prediction to deviation ratio (RPD) for the prediction of alpha-pinene were 0.976, 0.979, 0.305%, 0.317% and 6.826, respectively. In general, the results satisfactorily indicate that Vis-NIRS, with the appropriate algorithm, has promising results in determining myristicin and alpha-pinene on nutmeg from Aceh, Indonesia, as nondestructive measurement.