Color and appearance are perhaps the first attributes that attract us to a fruit or vegetable. Since the appearance of the product generally determines whether a product is accepted or rejected, measuring the color characteristics becomes an important task. To carry out the analysis of this key attribute for agriculture, it is recommended to use an artificial vision system to capture the images of the samples and then to process them by applying colorimetric routines to extract color parameters in an efficient and nondestructive manner, which makes it a suitable tool for a wide range of applications. The purpose of this chapter is to give an overview on recent development of image processing applied to color analysis from horticultural products, more specifically the practical usage of color image analysis in agriculture. As an example, quantitative values of color are extracted from Habanero Chili Peppers using image processing; the images from the samples were obtained using a desktop configuration of machine vision system. The material presented should be useful for students starting on the field, as well as for researchers looking for state-of-the-art studies and practical applications.