In recent years, the possibility to induce chemical transformations by using tunable plasmonic modes has opened the question of whether we can control or create chemical hot spots in these systems. This can be rationalized as the reactive analogue of the well-established concept of optical hot spots, which have drawn a great deal of attention to plasmonic nanostructures for their ability to circumvent the far-field diffraction limit of conventional optical elements. The parameters that determine the reactivity of plasmonic systems are deeply influenced by the dynamics and interplay of photons, plasmon-polaritons, carriers, phonons and molecular states. Although optical hot spots can be mainly defined by the geometry and permittivity of the nanostructures, the degrees of freedom influencing their photocatalytic properties appear to be much more numerous. These degrees of freedom can affect the reaction rates, the product selectivity or the spatial localization of a chemical reaction. In this Account, we show how the control of chemical hot spots can be achieved by carefully tuning the parameters that influence the cascade of events following the optical excitation of plasmonic modes in nanostructures. We discuss here a series of single photocatalyst techniques and ideas of how plasmonic nanoscale reactivity can be spatially mapped and imaged, and how the lifetime of hot and thermalized carriers can be altered by trap states on semiconductors and by metal-semiconductor interfaces. In addition, the tailored generation of non-thermal phonons in metallic nanostructures and their dissipation is shown as a promise to understand and exploit thermal photocatalysis at the nanoscale. Surpassing or enhancing each of these energy channels should enable to engineer solar nanometric photocatalysts. Nevertheless, the ultimate capability of a plasmonic photocatalyst to trigger a chemical reaction is correlated to its ability to navigate through, or even modify, the potential energy surface of a given chemical reaction. Here we reunite both worlds, the plasmonic photocatalysts and the molecular ones, identifying different energy transfer pathways and their influence on selectivity and efficiency of chemical reactions. We foresee that the migration from optical to chemical hot spots will greatly assist the understanding of ongoing plasmonic chemistry.