We present a new technique of broad-band heterodyne-detected sum frequency generation (HD-SFG) spectroscopy and demonstrate its high sensitivity allowing surface-selective measurements of vibrational spectra at submonolayer surface coverage, as low as a few percent of a monolayer. This was achieved without the help of surface enhancement phenomena, on a transparent dielectric substrate (water), and without introducing fluorescent labels, in fact, without utilizing any electronic resonances. Only the intrinsic vibrational transitions were employed for the detection of the analyte molecules (1-octanol). Unlike conventional (homodyne-detected) SFG spectroscopy, where the signal intensity decreases quadratically with decreasing surface coverage, in HD-SFG, the scaling is linear, and the signal is amplified by interference with a reference beam, significantly improving sensitivity and detection limits. At the same time, HD-SFG provides the phase as well as the amplitude of the signal and thus allows accurate subtraction of the non-resonant background--a common problem for surfaces with low concentrations of analyte molecules (i.e., weak resonant signals).

Vibrational sum frequency generation (VSFG) spectroscopy was used to study the nanoscale geometric effects on molecular conformation of dodecanethiol ligand on gold nanoparticles of varying size between 1.8 and 23 nm. By analyzing the CH3 and CH2 stretch transitions of dodecanethiol using the spectroscopic propensity rules for the SFG process, we observe the increase of the gauche defects in the alkyl chain of the ligand on the nanoparticle surface when the curvature approaches the size of the molecule ( approximately 1.6 nm). In contrast, linear infrared absorption and Raman spectra, governed by different selection rules, do not allow observation of the size-dependent conformational changes. The results are understood in terms of the geometric packing effect, where the curvature of the nanoparticle surface results in the increased conical volume available for the alkyl chain.