We present a versatile new instrument capable of measuring rovibrational transition frequencies of molecular ions with sub-MHz accuracy and precision. A liquid-nitrogen cooled positive column discharge cell, which can produce large column densities of a wide variety of molecular ions, is probed with sub-Doppler spectroscopy enabled by a high-power optical parametric oscillator locked to a moderate finesse external cavity. Frequency modulation (heterodyne) spectroscopy is employed to reduce intensity fluctuations due to the cavity lock, and velocity modulation spectroscopy permits ion-neutral discrimination. The relatively narrow Lamb dips are precisely and accurately calibrated using an optical frequency comb. This method is completely general as it relies on the direct measurement of absorption or dispersion of rovibrational transitions. We expect that this new approach will open up many new possibilities: from providing new benchmarks for state-of-the-art ab initio calculations to supporting astronomical observations to helping assign congested spectra by combination differences. Herein, we describe the instrument in detail and demonstrate its performance by measuring ten R-branch transitions in the ν2 band of H3(+), two transitions in the ν1 band of HCO(+), and the first sub-Doppler transition of CH5(+).