While non-Hermitian Hamiltonians have been experimentally realized in cold atom systems, it remains an outstanding open question of how to experimentally measure their complex energy spectra in momentum space for a realistic system with boundaries. The existence of non-Hermitian skin effects may make the question even more difficult to address given the fact that energy spectra for a system with open boundaries are dramatically different from those in momentum space; the fact may even lead to the notion that momentum-space band structures are not experimentally accessible for a system with open boundaries. Here, we generalize the widely used radio-frequency spectroscopy to measure both real and imaginary parts of complex energy spectra of a non-Hermitian quantum system. By weakly coupling the energy levels of a non-Hermitian system to auxiliary energy levels, we theoretically derive a formula showing that the decay of atoms on the auxiliary energy levels reflects the real and imaginary parts of energy spectra in momentum space. We further prove that measurement outcomes are independent of boundary conditions in the thermodynamic limit, providing strong evidence that the energy spectrum in momentum space is experimentally measurable. We finally apply our non-Hermitian absorption spectroscopy protocol to the Hatano-Nelson model and non-Hermitian Weyl semimetals to demonstrate its feasibility.