An automated quality control (QC) system is essential to ensure streamlined head computed tomography (CT) scan interpretations that do not affect subsequent image analysis. Such a system is advantageous compared to current human QC protocols, which are subjective and time-consuming. In this work, we aim to develop a deep learning-based framework to classify a scan to be of usable or unusable quality. Supervised deep learning models have been highly effective in classification tasks, but they are highly complex and require large, annotated data for effective training. Additional challenges with QC datasets include -1) class-imbalance -usable cases far exceed the unusable ones and 2) weak-labels -scan level labels may not match slice level labels. The proposed framework utilizes these weak labels to augment a standard anomaly detection technique. Specifically, we proposed a hybrid model that consists of a variational autoencoder (VAE) and a Siamese Neural Network (SNN). While the VAE is trained to learn how usable scans appear and reconstruct an input scan, the SNN compares how similar this input scan is to its reconstruction and flags the ones that are less similar than a threshold. The proposed method is more suited to capture the differences in non-linear feature structure between the two classes of data than typical anomaly detection methods that depend on intensity-based metrics like root mean square error (RMSE). Comparison with state-of-the-art anomaly detection methods using multiple classification metrics establishes superiority of the proposed framework in flagging inferior quality scans for review by radiologists, thus reducing their workload and establishing a reliable and consistent dataflow.