CNN (Convolutional Neural Network) is one of the type of Deep Learning algorithm that can accept input in the form of images, or objects such as fruit, animals, office equipment, and so on. This CNN works is inspired by the neuron system in humans based on visual perception. The CNN model can be implemented into the Android system using TFLite. TFLite is a Machine Learning library specially designed for Android devices to create Machine Learning models in small binary and low latency. This can make it easier for Machine Learning researchers to use smartphones to develop the research system. Current technological advances make the fruit industry no longer use the manual fruit sorting process carried out by humans. The process of sorting fruit that is fit for consumption (fresh) and unfit for consumption (rotten) can reach hundreds and thousands, so it needs the help of machines to increase work speed. This fruit sorting process can be circumvented by creating a Machine Learning model that can classify fresh and rotten fruit which will then be processed by the system based on the desired output requirements. If the model can perform the classification well, then the fruit industry can take steps to make an automatic sorter by implementing the model into its system. This study aims to create a model with the CNN (Convolutional Neural Network) Algorithm and design a prototype system that can classify fresh and rotten fruit (red apples, bananas, and mandarin oranges) and apply the resulting model using TFLite on an Android Smartphone. The results indicated that the resulting prototype model can be used by users in the Android system. The results of the training model obtained an accuracy value of 98.20%. Meanwhile, the results of testing showed that the model can classify images from the same or different sources by 91.65% from 1090 images. These results indicate that the model is included in the excellent classification category.