A rotating substrate is essential for achieving highquality and uniform growth of gallium nitride (GaN) thin films in molecular beam epitaxy (MBE). By rotating the substrate during growth, the deposited material can be distributed more uniformly across the surface, resulting in a more uniform and defect-free film. However, the substrate may stop rotating during fabrication, and it is a challenge for engineers to detect the rotation of the substrate using information from reflection high-energy electron diffraction (RHEED). The current method for analyzing rotation using RHEED data is manual, time-consuming, and error-prone. Therefore, this research proposes an automated machine learning-based rotation error detection (RED) model to determine wafer rotation and then alert the engineers in case of an anomaly. The proposed approach consists of (1) ensemble learning models such as random forest classifier and extreme gradient boosting (XGBoost) for rotation error detection, (2) prepared novel intensity data from RHEED videos, (3) comparison to select the best ensemble learning model at a small scale, ( 4) training XGBoost at a large scale, (5) deployment in production using MLflow, (6) development of a REST application programming interface (API) for inference, and (7) integrated KakaoTalk API to generate and send a detailed alert message to engineers so that they can take appropriate actions in time. The experimental results achieved F1 scores for normal and rotational errors of 98.9 and 83.1%, respectively. The proposed method can make a valuable contribution to epitaxial wafer growth by enhancing production efficiency, improving product quality, and reducing costs.