Volunteer potato is an increasing problem in crop rotations where winter temperatures are often not cold enough to kill tubers leftover from harvest. Poor control, as a result of high labor demands, causes diseases like Phytophthora infestans to spread to neighboring fields. Therefore, automatic detection and removal of volunteer plants is required. In this research, an adaptive Bayesian classification method has been developed for classification of volunteer potato plants within a sugar beet crop. With use of ground truth images, the classification accuracy of the plants was determined. In the non-adaptive scheme, the classification accuracy was 84.6 and 34.9% for the constant and changing natural light conditions, respectively. In the adaptive scheme, the classification accuracy increased to 89.8 and 67.7% for the constant and changing natural light conditions, respectively. Crop row information was successfully used to train the adaptive classifier, without having to choose training data in advance.