From the hundreds of gamma-ray pulsars known (Atwood et al. 2009), only a handful show non-thermal X-ray pulsations (Kuiper & Hermsen 2015). Instead, nine objects pulse in non-thermal X-rays but lack counterparts at higher energies. Here, we present a physical model for the non-thermal emission of pulsars above 1 keV. With just four physical parameters, we fit the spectrum of the gamma/X-ray pulsars along seven orders of magnitude. We find that all detections can be encompassed in a continuous variation of the model parameters, and pose that their values could likely relate to the closure mechanism operating in the accelerating region. The model explains the appearance of sub-exponential cutoffs at high energies as a natural consequence of synchro-curvature dominated losses, unveiling that curvature-only emission may play a relatively minor role -if any-in the spectrum of most pulsars. The model also explains the flattening of the X-ray spectra at soft energies as a result of propagating particles being subject to synchrotron losses all along their trajectories. Using this model, we show how observations in gamma-rays can predict the detectability of the pulsar in X-rays, and viceversa.Curvature and synchrotron emission from particles accelerated in magnetospheric gaps or reconnection (see e.g., Cheng et al. 1986a,b;Romani 1996;Zhang & Cheng 1997;Hirotani & Shibata 1999;Muslimov & Harding 2003;Dyks & Rudak 2003;Kalapotharakos et al. 2012;Philippov & Spitkovsky 2017) has been thought to be behind non-thermal pulsations. However, a simplified, unifying interpretation of pulsar spectra is still lacking. Ab initio modelling with sophisticated electrodynamics are not yet focusing on precisely reproducing spectra, and studies of specific pulsars, with different degrees of complexity, usually produce unsatisfactory fits and are based on a significant number of case-by-case assumptions (see e.g., Kalapotharakos et al. 2017;Hirotani 2015;Takata & Cheng 2017). As a result, we normally fit observational spectra just with phenomenological functions: a power law with a cutoff in gamma-rays, or a log parabola from X-rays up. This status deters population analysis as well as the extraction of physical characteristics.The basis of the physical model we use here has been presented in a series of papers deal- (Viganò et al. 2015a,b), and with a systematic analysis of the gamma-ray results obtained by Fermi-LAT (Viganò & Torres 2015;Viganò et al. 2015c). The model we propose is conceptually simple: it goes from particle dynamics to radiation. It is based on the assumption, see e.g., Chiang & Romani (1994); Zhang & Cheng (1997); Kalapotharakos et al. (2012), that somewhere near the light cylinder of a pulsar having period P and period derivativeṖ, there is a gap with a significant component of electric field parallel to the magnetic lines. This field accelerates particles. Once a particle enters into this gap (for instance, by being created there in a pair-production process) the model follows its time-evolution solving the equat...