Molecular recognition of carbohydrates plays an important role in nature. The aggregation of the smallest sugar, glycolaldehyde, was studied in a conformer-selective manner using high-resolution rotational spectroscopy. Two different dimer structures were observed. The most stable conformer reveals C2 -symmetry by forming two intermolecular hydrogen bonds, giving up the strong intramolecular hydrogen bonds of the monomers and thus showing high hydrogen bond selectivity. By analyzing the spectra of the (13) C and (18) O isotopologues of the dimer in natural abundance, we could precisely determine the heavy backbone structure of the dimer. Comparison to the monomer structure and the complex with water provides insight into intermolecular interactions. Despite hydrogen bonding being the dominant interaction, precise predictions from quantum-chemical calculations highly rely on the consideration of dispersion.