Context. An important benchmark for current observational techniques and theoretical modeling of exoplanet atmospheres is the transit of Venus (ToV). Stellar activity and, in particular, convection-related surface structures, potentially cause fluctuations that can affect the transit light curves. Surface convection simulations can help interpreting the ToV as well as other transits outside our solar system. Aims. We used the realistic three-dimensional (3D) radiative hydrodynamical (RHD) simulation of the Sun from the Stagger-grid and synthetic images computed with the radiative transfer code Optim3D to predict the transit of Venus (ToV) in 2004 that was observed by the satellite ACRIMSAT. Methods. We computed intensity maps from the RHD simulation of the Sun and produced a synthetic stellar disk image as an observer would see, accounting for the center-to-limb variations. The contribution of the solar granulation was considered during the ToV. We computed the light curve and compared it to the ACRIMSAT observations as well as to light curves obtained with solar surface representations carried out using radial profiles with different limb-darkening laws. We also applied the same spherical tile imaging method as used for RHD simulation to the observations of center-to-limb solar granulation with Hinode. Results. We explain ACRIMSAT observations of 2004 ToV and show that the granulation pattern causes fluctuations in the transit light curve. We compared different limb-darkening models to the RHD simulation and evaluated the contribution of the granulation to the ToV. We showed that the granulation pattern can partially explain the observed discrepancies between models and data. Moreover, we found that the overall agreement between real and RHD solar granulation is good, either in terms of depth or ingress/egress slopes of the transit curve. This confirms that the limb-darkening and granulation pattern simulated in 3D RHD of the Sun represent well what is imaged by Hinode. In the end, we found that the contribution of the Venusean aureole during ToV is ∼10 −6 times less intense than the solar photosphere, and thus, accurate measurements of this phenomena are extremely challenging. Conclusions. The prospects for planet detection and characterization with transiting methods are excellent with access to large a amount of data for stars. Being able to consistently explain the data of 2004 ToV is a new step forward for 3D RHD simulations, which are becoming essential for detecting and characterizing exoplanets. They show that granulation has to be considered as an intrinsic uncertainty (as a result of stellar variability) on precise measurements of exoplanet transits of, most likely, planets with small diameters. In this context, it is mandatory to obtain a comprehensive knowledge of the host star, including a detailed study of the stellar surface convection.