Single photon detectors play a key role across several basic science and technology applications. While progress has been made in improving performance, single photon detectors that can maintain high performance while also resolving the photon frequency are still lacking. By means of quantum simulations, we show that nanoscale elements cooperatively interacting with the photon field in a photodetector architecture allow to simultaneously achieve high efficiency, low jitter, and high frequency resolution. We discuss how such cooperative interactions are essential to reach this performance regime, analyzing the factors that impact performance and trade-offs between metrics. We illustrate the potential performance for frequency resolution over a 1 eV bandwidth in the visible range, indicating near perfect detection efficiency, jitter of a few hundred femtoseconds, and frequency resolution of tens of meV. Finally, a potential physical realization of such an architecture is presented based on carbon nanotubes functionalized with quantum dots.