The classical color constancy algorithms concentrate only on the color of a grey surface to estimate the light color and to white balance the image. In this paper, we show that the quality of the whole process can be clearly improved by predicting and correcting the colors of a set of reference surfaces. The ground truth of the surface color under white light can be easily obtained with a set of images acquired by the considered camera under sun light. Thus, we design a deep network to predict the colors of the reference surfaces of a color checker as if it had been in the scene at acquisition time. We show that our solution improves the two steps of the color constancy process on 9 datasets and we claim that being able to synthetically insert a color chart in any image can help for many other tasks.