As a test-bed for future investigations of directly imaged terrestrial exoplanets, we present the recovery of the surface components of the Earth from multi-band diurnal light curves obtained with the EPOXI spacecraft. We find that the presence and longitudinal distribution of ocean, soil and vegetation are reasonably well reproduced by fitting the observed color variations with a simplified model composed of a priori known albedo spectra of ocean, soil, vegetation, snow and clouds. The effect of atmosphere, including clouds, on light scattered from surface components is modeled using a radiative transfer code. The required noise levels for future observations of exoplanets are also determined. Our model-dependent approach allows us to infer the presence of major elements of the planet (in the case of the Earth, clouds and ocean) with observations having S/N 10 in most cases and with high confidence if S/N 20. In addition, S/N 100 enables us to detect the presence of components other than ocean and clouds in a fairly model-independent way. Degradation of our inversion procedure produced by cloud cover is also quantified. While cloud cover significantly dilutes the magnitude of color variations compared to the cloudless case, the pattern of color changes remains. Therefore, the possibility of investigating surface features through light curve fitting remains even for exoplanets with cloud cover similar to the Earth's.