For the development of the agricultural industry, the urgent task of designing robots capable of efficiently harvesting crops is being set. To do this, it is necessary to ensure the high quality of the algorithm of the computer vision system for determining the boundaries of fruits and the degree of their ripeness. Differences in weather conditions and illumination of a fruit tree have a significant impact on the operation of such an algorithm. It is shown that due to the introduction of neural networks into machine vision systems, the speed of fruit detection has significantly increased. The development of robots for harvesting fruits will allow replacing heavy physical labor of a person, as well as reducing the percentage of crop shortage.