Indonesia has an abundance and variety of fruit commodities. Referring to BPS, in 2019 the fruit production reached 22 million tons or increase 5% compared to 2018. However, with such a large production volume, most of the fruit inspection process in Indonesia still performed with human intervention. This process is very labor intensive, time consuming, and prone to inconsistencies and inaccuracies. The automation process using computer vision technology is expected to eliminate manual process therefore reducing costs, increasing efficiency and accuracy. In this study, YOLOv4 algorithm was applied to detect banana ripeness automatically. The training process is carried out using 369 banana images which are divided into two classes and the testing process is carried out on videos that are captured in real-time. Based on the research results, the best average accuracy rate is 87.6% and the video processing speed is 5 FPS (frames per second) using a single-GPU architecture.