Background: The COVID-19 pandemic has claimed numerous lives in the last three years. With new variants emerging every now and then, the world is still battling with the management of COVID-19. Purpose: To utilize a deep learning model for the automatic detection of severity scores from chest CT scans of COVID-19 patients and compare its diagnostic performance with experienced human readers. Methods: A deep learning model capable of identifying consolidations and ground-glass opacities from the chest CT images of COVID-19 patients was used to provide CT severity scores on a 25-point scale for definitive pathogen diagnosis. The model was tested on a dataset of 469 confirmed COVID-19 cases from a tertiary care hospital. The quantitative diagnostic performance of the model was compared with three experienced human readers. Results: The test dataset consisted of 469 CT scans from 292 male (average age: 52.30) and 177 female (average age: 53.47) patients. The standalone model had an MAE of 3.192, which was lower than the average radiologists' MAE of 3.471. The model achieved a precision of 0.69 [0.65, 0.74] and an F1 score of 0.67 [0.62, 0.71], which was significantly superior to the average reader precision of 0.68 [0.65, 0.71] and F1 score of 0.65 [0.63, 0.67]. The model demonstrated a sensitivity of 0.69 [95% CI: 0.65, 0.73] and specificity of 0.83 [95% CI: 0.81, 0.85], which was comparable to the performance of the three human readers, who had an average sensitivity of 0.71 [95% CI: 0.69, 0.73] and specificity of 0.84 [95% CI: 0.83, 0.85]. Conclusion: The AI model provided explainable results and performed at par with human readers in calculating CT severity scores from the chest CT scans of patients affected with COVID-19. The model had a lower MAE than that of the radiologists, indicating that the CTSS calculated by the AI was very close in absolute value to the CTSS determined by the reference standard.