Plasmonic hot carrier devices extract excited carriers from metal nanostructures before equilibration, and have the potential to surpass semiconductor light absorbers. However their efficiencies have so far remained well below theoretical limits, which necessitates quantitative prediction of carrier transport and energy loss in plasmonic structures to identify and overcome bottlenecks in carrier harvesting. Here, we present a theoretical and computational framework, Non-Equilibrium Scattering in Space and Energy (NESSE), to predict the spatial evolution of carrier energy distributions that combines the best features of phase-space (Boltzmann) and particle-based (Monte Carlo) methods. Within the NESSE framework, we bridge first-principles electronic structure predictions of plasmon decay and carrier collision integrals at the atomic scale, with electromagnetic field simulations at the nano-to mesoscale. Finally, we apply NESSE to predict spatially-resolved energy distributions of photo-excited carriers that impact the surface of experimentally realizable plasmonic nanostructures at length scales ranging from tens to several hundreds of nanometers, enabling first-principles design of hot carrier devices.Surface plasmon resonances shrink optics to the nano scale, facilitating strong focusing and localized absorption of light [1][2][3][4][5]. Decay of plasmons generates energetic electrons and holes in the material that can be exploited for applications including photodetection, imaging and spectroscopy [6-12], photonic energy conversion, and photocatalysis [13][14][15][16][17][18]. However, these applications require carriers that retain a significant fraction of their energy absorbed from the plasmon, which is typically two orders of magnitude larger than the thermal energy scale. Experimentally, the energy distributions of hot carriers that critically impact their efficiency of collection cannot be measured directly, but must instead be inferred indirectly from optical response in pump-probe measurements, [19][20][21] from photo-current measurements, [22,23] or from redox-reaction chemical markers.[24] This critically necessitates theoretical prediction of charge transport in metal nanostructures far from equilibrium, which presents a major challenge for current computational methods [25][26][27][28].In extremely small nano-scale systems, electron dynamics require a full quantum mechanical treatment, and several classes of techniques have been developed for quantum transport simulations. In diagrammatic many-body perturbation theory, quantum transport can be described using the non-equilibrium Greens function (NEGF) formalism [29], which has been applied extensively to electron transport in molecular junctions [30]. atoms in rarefied gases [31], nanoscale metal interconnects [32], and small plasmonic nanoparticles [33]. Open quantum system approaches applied to photons have simi- * prineha@seas.harvard.edu † sundar@rpi.edu larly enabled efficient prediction of retardation and radiative effects on plasmon resonance...