Vibrational sum-frequency generation (VSFG) has become a powerful tool for investigating the molecular structure of surfaces and interfaces. One aspect of the technique that has been overlooked is the source and nature of the nonresonant contribution to the measured signal. If not properly accounted for, interaction between the resonant and nonresonant responses can lead to misinterpretation of the resonant signals. The nonresonant response itself also carries important information about the system under study. This Perspective reviews some of the experimental and post-processing techniques that have been developed to isolate the nonresonant response. The focus of such approaches has generally been on the resonant signal, meaning more can be done to learn about the nonresonant response. Some examples of information carried by the nonresonant response are also presented, again showing that there is more to do to understand this signal. The nonresonant sum-frequency generation response is an important component of VSFG measurement, and the VSFG community needs to learn to better understand what it is telling us about our samples.