Coherent Elastic Neutrino-Nucleus Scattering (CEνNS) is a Standard Model process that, although predicted for decades, has only been detected recently by the COHERENT collaboration. Now that CEνNS has been discovered, it provides a new probe for physics beyond the Standard Model. We study the potential to probe New Physics with CEνNS through the use of low temperature bolometers at a reactor source. We consider contributions to CEνNS due to a neutrino magnetic moment (NMM), Non-Standard Interactions (NSI) that may or may not change flavor, and simplified models containing a massive scalar or vector mediator. Targets consisting of Ge, Zn, Si, CaWO 4 , and Al 2 O 3 are examined. We show that by reaching a percentage-level precision measurement on the CEνNS energy spectrum down to O(10) eV, forthcoming experiments will improve by two orders of magnitude both the CEνNS-based NMM limit and the search for new massive mediators. Additionally, we demonstrate that such dedicated low-threshold CEνNS experiments will lead to unprecedented constraints on NSI parameters (particularly when multiple targets are combined) which will have major implications for the global neutrino physics program.