Fruit is one of the nutritional needs for the body that must be met. But with a note, these nutrients will be obtained from fruit that is still fresh. The definition of fresh fruit itself is fruit that can be consumed directly and does not require any further processing. There are many ways to select and differentiate between fresh fruit and bad fruit and in general direct observations can be made. But over time, there are several other ways to observe fruit freshness using existing technology. Where one of them is by optimizing deep learning and machine learning. This detection and classification system was created using a deep learning method using the YOLOv5 algorithm which can detect in real-time the types of apples, bananas and oranges. We use image datasets for each of these fruits for fresh fruit and rotten fruit, a total of 1200 images for train data and 330 images for validation data and 6 images for test data. Based on the tests that have been carried out with training data, along with validation data, and test data using the YOLOv5 algorithm, it can be concluded that this detection method can recognize objects consistently with a high degree of accuracy. This can be proven at the level of accuracy which reaches an accuracy rate of 90%.