Edge illumination (EI) is an X-ray imaging technique that, in addition to conventional absorption contrast, provides refraction and scatter contrast. It relies on an absorption mask in front of the sample that splits the X-ray beam into beamlets, which hits a second absorption mask positioned in front of the detector. The sample mask is then shifted in multiple steps with respect to the detector mask, thereby measuring an illumination curve per detector element. The width, position, and area of this curve estimated with and without the sample in the beam is then compared, which ultimately provides absorption, refraction, and scatter contrast for each detector pixel. From the obtained contrast sinograms, three contrast tomograms can be computed. In summary, conventional EI relies on a two-stage process comprised of a computational and time intensive contrast retrieval process, followed by tomographic reconstruction. In this work, a novel joint reconstruction method is proposed, which utilizes a combined forward model to reconstruct the three contrasts simultaneously, without the need for an intermediate contrast retrieval step. Compared to the state-of-the-art, this approach reduces reconstruction times, as the retrieval step is skipped and allows a much more flexible acquisition scheme, as there is no need to sample a full illumination curve at each projection angle. The proposed method is shown to improve reconstruction quality on subsampled datasets, enabling the reconstruction of three contrasts from single-shot datasets.