In-line digital holography is a simple yet powerful tool to image absorbing and/or phase objects. Nevertheless, the loss of the phase of the complex wavefront on the sensor can be critical in the reconstruction process. The simplicity of the setup must thus be counterbalanced by dedicated reconstruction algorithms, such as inverse approaches, in order to retrieve the object from its hologram. In the case of simple objects for which the diffraction pattern produced in the hologram plane can be modeled using few parameters, a model fitting algorithm is very effective. However, such an approach fails to reconstruct objects with more complex shapes, and an image reconstruction technique is then needed. The improved flexibility of these methods comes at the cost of a possible loss of reconstruction accuracy. In this work, we combine the two approaches (model fitting and regularized reconstruction) to benefit from their respective advantages. The sample to be reconstructed is modeled as the sum of simple parameterized objects and a complex-valued pixelated transmittance plane. These two components jointly scatter the incident illumination, and the resulting interferences contribute to the intensity on the sensor. The proposed hologram reconstruction algorithm is based on alternating a model fitting step and a regularized inversion step. We apply this algorithm in the context of fluid mechanics, where holograms of evaporating droplets are analyzed. In these holograms, the high contrast fringes produced by each droplet tend to mask the diffraction pattern produced by the surrounding vapor wake. With our method, the droplet and the vapor wake can be jointly reconstructed.