Automation in every part of life has become a frequent situation because of the rapid advancement of technology, mostly driven by AI technology, and has helped facilitate improved decision-making. Machine learning and the deep learning subset of AI provide machines with the capacity to make judgments on their own through a continuous learning process from vast amounts of data. To decrease human mistakes while making critical choices and to improve knowledge of the game, AI-based technologies are now being implemented in numerous sports, including cricket, football, basketball, and others. Out of the most globally popular games in the world, cricket has a stronghold on the hearts of its fans. A broad range of technologies are being discovered and employed in cricket by the grace of AI to make fair choices as a method of helping on-field umpires because cricket is an unpredictable game, anything may happen in an instant, and a bad judgment can dramatically shift the game. Hence, a smart system can end the controversy caused just because of this error and create a healthy playing environment. Regarding this problem, our proposed framework successfully provides an automatic no-ball detection with 0.98 accuracy which incorporates data collection, processing, augmentation, enhancement, modeling, and evaluation. This study starts with collecting data and later keeps only the main portion of bowlers’ end by cropping it. Then, image enhancement technique are implied to make the image data more clear and noise free. After applying the image processing technique, we finally trained and tested the optimized CNN. Furthermore, we have increased the accuracy by using several modified pretrained model. Here, in this study, VGG16 and VGG19 achieved 0.98 accuracy and we considered VGG16 as the proposed model as it outperformed considering recall value.