Online exams are growing increasingly popular in organizations and educational institutes because they are more flexible and cost-effective than conventional paper-based exams. When face-to-face exams are not possible, such as during floods, unexpected situations, or pandemics like COVID-19, this exam mod has become even more popular and important. However, online exams may have difficulties, such as the need for a reliable internet connection and the possibility of cheating. Because there is no human supervisor present to monitor the exam, so cheating is a major concern. The environment employed for the online exams ensures that every student finalizes the evaluation process without using any type of cheating. This study investigates the detection and recognition of unusual behavior in an academic setting, such as online exams, to prevent students from cheating or engaging in unethical behavior. After consulting with experts and reviewing the online exam held in Covid-19 and other online exams, selected the four most common cheating activities found in the online exam. The study extracts key frames using motion-based frame extraction techniques before employing advanced deep learning techniques with various convolutional neural network configurations. This study presents several deep learning-based models that analyze the video exam to classify four categories of cheating. This method extracts key frames from a video sequence/stream based on human motion. This research developed a real dataset of cheating behaviors and conducted comprehensive experiments with pre-trained and suggested deep-learning models. When evaluated using standard performance criteria, the You Only Look Once (YOLO)v5 model outperforms other pre-trained and fine-tuned approaches for detecting unusual activity.