Potato is one of the widely used products all over the world that has numerous nutritional properties. Similar to other crops, different weeds grow along with potatoes in agricultural fields. These weeds reduce the performance of crops due to competing with them to absorb water, light, and nutrients from soil. Accordingly, in this study, a machine vision system with the hybrid artificial neural network-ant colony algorithm (ANN-ACO) classifier was developed for a site-specific spraying considering the weed type. Potato plant and three weed types including Chenopodium album, Polygonum aviculare L., and Secale cereale L. were used in this study. A digital camera (SAMSUNG WB151F (CCD, 14.2 MP, 30f/s) was placed in the center of the video acquisition system. The distance between plants and the digital camera was fixed at 40 cm. For video acquisition, only lamps of white LED with a light intensity of 327 lux were selected. For filming in order to evaluate the proposed system, a 4-hectare area of Agria potato fields in Kermanshah-Iran (longitude: 7.03°E; latitude: 4.22°N) was selected. Employing the Gamma test, among 31 features, 5 features (Luminance and Hue corresponding to YIQ color space, Autocorrelation, Contrast, and Correlation) were selected. The correct classification accuracy for testing and training data using three classifiers of the hybrid ANN-ACO, radial basis function (RBF) artificial neural network, and Discriminant analysis (DA) was 99.6% and 98.13%, 97.24% and 91.23%, and 69.8% and 70.8%, respectively. The results show that the accuracy of DA statistical method is much lower than that of the hybrid ANN-ACO classifier. Consequently, the results of the present study can be used in machine vision system for the optimum spraying of herbicides.