In this study, an image‐based measurement system was developed for human facial skin colour, involving the development of a digital imaging system, collection of facial skin colour from 60 human subjects, generation of different colour characterization models, and performance evaluation. The factors that affect facial skin colour characterization, including different training datasets (two colour charts and the collected facial skin colour dataset), mathematical mapping methods (linear transformation, polynomial regression, root‐polynomial regression and neural network) and camera image formats (JPG and RAW), were investigated and quantified not only by the conventional method of CIELAB colour difference, but also two newly introduced measures, facial colour contrast and skin colour gamut. The results indicate that the RAW image format for camera digital signals gave a more stable performance than the JPG format images, and the higher order polynomial regression with good predictive accuracy in terms of CIELAB colour difference did not perform well for the whole facial image. It is suggested to evaluate the model performance using the colour of both specific facial positions and the overall facial skin colour. Our comparative analysis in this study provides useful guidance for determining the colour characterization model for facial skin.