Optical heterodyne detection-based spectrometers are attractive due to their relatively simple construction and ultra-high resolution. Here we demonstrate a proof-ofprinciple heterodyne spectrometer which has picometer resolution and quantum-limited sensitivity. Moreover, we report a generalized quantum limit of detecting broadband multi-mode light using heterodyne detection, which provides a sensitivity limit on a heterodyne detection-based optical spectrometer. We then compare this sensitivity limit to several spectrometer types and dim light sources of interest, such as, spontaneous parametric downconversion, Raman scattering, and spontaneous four-wave mixing. We calculate the heterodyne spectrometer is significantly less sensitive than a single-photon detector and unable to detect these dim light sources under most circumstances.Notice: This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).