Edge illumination is a phase sensitive X-ray imaging technique, compatible with lab-based X-ray sources. Phase information can be retrieved by displacing two masks relative to each other with the object positioned in between. Phase retrieval then allows the three contrasts, absorption, refraction and dark field, to be individually reconstructed with FBP. In this work, a novel joint reconstruction method is proposed using a combined forward model, allowing the three contrasts to be reconstructed simultaneously, without a retrieval step. This allows for more freedom in the acquisition scheme. The proposed objective function is minimized using a split Barzilai-Borwein gradient method. Improved convergence speed and reconstruction quality on an experimental dataset, compared to existing methods, is shown.