Abstract-Silicon drift detectors (SDDs) revolutionized spectroscopy in fields as diverse as geology and dentistry. For a subset of experiments at ultra-fast, x-ray free-electron lasers (FELs), SDDs can make substantial contributions. Often the unknown spectrum is interesting, carrying science data, or the background measurement is useful to identify unexpected signals. Many measurements involve only several discrete photon energies known a priori, allowing single event, Bayesian decomposition of pile-up and spectroscopic photon counting. We designed a pulse function (a combination of gradual step and exponential decay function) and demonstrated that for individual pulses the signal amplitude, peaking time, and pulse amplitude are interrelated and at short peaking times the pulse amplitude is not an optimal estimator for signal amplitude; instead, the signal amplitude and peaking time are obtained for each pulse by fitting, thus removing the need for pulse shaping. Avoiding pulse shaping reduced peaking times to tens of nanoseconds, resulting in reduced pulse pile-up and allowing decomposition of remaining pulse pile-up at photon separation times down to 100 ns while yielding time-of-arrival information with precision of 10 nanoseconds. At pulsed sources or high photon rates, photon pile-up still occurs. We showed that the area of one photon peaks is not suitable for estimating high photon rates while pile-up spectrum fitting is relatively simple and preferable to pile-up spectrum deconvolution. We developed a photon pile-up model for constant intensity sources, extended it to variable intensity sources (typical for FELs) and used it to fit a complex pileup spectrum, demonstrating its accuracy. Based on the pile-up model, we developed a Bayesian pile-up decomposition method that allows decomposing pile-up of single events with up to 6 photons from 6 monochromatic lines with 99% accuracy. The usefulness of SDDs will continue into the x-ray FEL era of science. Their successors, the ePixS hybrid pixel detectors, already offer hundreds of pixels, each with similar performance to an SDD, in a compact, robust and affordable package.