The conformational landscapes of two highly flexible monosaccharide derivatives, namely phenyl β-D-glucopyranoside (ph-β-glu) and 4-(hydroxymethyl)phenyl β-D-glucopyranoside, also commonly known as gastrodin, were explored using a combined experimental and theoretical approach. For the infrared, Raman, and the associated vibrational optical activity (VOA), i.e., vibrational circular dichroism and Raman optical activity, experiments of these two compounds in DMSO and in water were carried out. Extensive and systematic conformational searches were performed using a recently developed conformational searching tool called CREST (conformer-rotamer ensemble sampling tool) in the two solvents. Fourteen and twenty-four low-energy conformers were identified at the DFT level for ph-β-glu and gastrodin, respectively. The spectral simulations of individual conformers were done at the B3LYP-D3BJ/def2-TZVPD level with the polarizable continuum model of the solvents. The VOA spectral features exhibit much higher specificity to conformational differences than their parent infrared and Raman. The excellent agreements achieved between the experimental and simulated VOA spectra allow for the extraction of experimental conformational distributions of these two carbohydrates in solution directly. The experimental percentage abundances based on the hydroxymethyl (at the pyranose ring) conformations G+, G-, and T for ph-β-glu were obtained to be 15%, 75%, and 10% in DMSO and 53%, 40%, and 7% in water, respectively, in comparison to the previously reported gas phase values of 68%, 25%, and 7%, highlighting the important role of solvents in conformational preferences. The corresponding experimental distributions for gastrodin are 56%, 22%, and 22% in DMSO and 70%, 21%, and 9% in water.