Background In developing countries like Indonesia, limited resources for routine mass Coronavirus Disease 2019 (COVID-19) RT-PCR testing among healthcare workers leave them with a heightened risk of late detection and undetected infection, increasing the spread of the virus. Accessible and accurate methodologies must be developed to identify COVID-19 positive healthcare workers. This study aimed to investigate the application of machine learning classifiers to predict the risk of COVID-19 positivity in high-risk populations where resources are limited and accessibility is desired. Methods Two sets of models were built: one both trained and tested on data from healthcare workers in Jakarta and Semarang, and one trained on Jakarta healthcare workers and tested on Semarang healthcare workers. Models were assessed by the area under the receiver-operating-characteristic curve (AUC), average precision (AP), and Brier score (BS). Shapley additive explanations (SHAP) were used to analyze feature importance. 5,394 healthcare workers were included in the final dataset for this study. Results For the full model, the voting classifier composed of random forest and logistic regression was selected as the algorithm of choice and achieved training AUC (mean [Standard Deviation (SD)], 0.832 [0.033]) and AP (mean [SD], 0.476 [0.042]) and was high performing during testing with AUC and AP of 0.753 and 0.504 respectively. A voting classifier composed of a random forest and a XGBoost classifier was best performing during cross-validation for the Jakarta model, with AUC (mean [SD], 0.827 [0.023]), AP (mean [SD], 0.461 [0.025]). The performance when testing on the Semarang healthcare workers was AUC of 0.725 and AP of 0.582. Conclusions Our models yielded high predictive performance and can be used as an alternate COVID-19 screening methodology for healthcare workers in Indonesia, although the low adoption rate by partner hospitals despite its usefulness is a concern.

Background: In developing countries like Indonesia, limited resources for routine mass Coronavirus Disease 2019 (COVID-19) RT-PCR testing among healthcare workers leave them with a heightened risk of late detection and undetected infection, increasing the spread of the virus. Accessible and accurate methodologies must be developed to identify COVID-19 positive healthcare workers. This study aimed to investigate the application of machine learning classifiers to predict the risk of COVID-19 positivity in high-risk populations where resources are limited and accessibility is desired. Methods: Two sets of models were built: one both trained and tested on data from healthcare workers in Jakarta and Semarang, and one trained on Jakarta healthcare workers and tested on Semarang healthcare workers. Models were assessed by the area under the receiver-operating-characteristic curve (AUC), average precision (AP), and Brier score (BS). Shapley additive explanations (SHAP) were used to analyze feature importance. 5,394 healthcare workers were included in the final dataset for this study. Results: For the full model, the voting classifier composed of random forest and logistic regression was selected as the algorithm of choice and achieved training AUC (mean [Standard Deviation (SD)], 0.832 [0.033]) and AP (mean [SD], 0.476 [0.042]) and was high performing during testing with AUC and AP of 0.753 and 0.504 respectively. A voting classifier composed of a random forest and a XGBoost classifier was best performing during cross-validation for the Jakarta model, with AUC (mean [SD], 0.827 [0.023]), AP (mean [SD], 0.461 [0.025]). The performance when testing on the Semarang healthcare workers was AUC of 0.725 and AP of 0.582. Conclusions: Our models yielded high predictive performance and can be used as an alternate COVID-19 screening methodology for healthcare workers in Indonesia, although the low adoption rate by partner hospitals despite its usefulness is a concern.

The COVID-19 pandemic poses a heightened risk to health workers, especially in low- and middle-income countries such as Indonesia. Due to the limitations to implementing mass RT-PCR testing for health workers, high-performing and cost-effective methodologies must be developed to help identify COVID-19 positive health workers and protect the spearhead of the battle against the pandemic. This study aimed to investigate the application of machine learning classifiers to predict the risk of COVID-19 positivity (by RT-PCR) using data obtained from a survey specific to health workers. Machine learning tools can enhance COVID-19 screening capacity in high-risk populations such as health workers in environments where cost is a barrier to accessibility of adequate testing and screening supplies. We built two sets of COVID-19 Likelihood Meter (CLM) models: one trained on data from a broad population of health workers in Jakarta and Semarang (full model) and tested on the same, and one trained on health workers from Jakarta only (Jakarta model) and tested on an independent population of Semarang health workers. The area under the receiver-operating-characteristic curve (AUC), average precision (AP), and the Brier score (BS) were used to assess model performance. Shapley additive explanations (SHAP) were used to analyze feature importance. The final dataset for the study included 3979 health workers. For the full model, the random forest was selected as the algorithm of choice. It achieved cross-validation mean AUC of 0.818 ± 0.022 and AP of 0.449 ± 0.028 and was high performing during testing with AUC and AP of 0.831 and 0.428 respectively. The random forest model was well-calibrated with a low mean brier score of 0.122 ± 0.004. A random forest classifier was the best performing model during cross-validation for the Jakarta dataset, with AUC of 0.824 ± 0.008, AP of 0.397 ± 0.019, and BS of 0.102 ± 0.007, but the extra trees classifier was selected as the model of choice due to better generalizability to the test set. The performance of the extra trees model, when tested on the independent set of Semarang health workers, was AUC of 0.672 and AP of 0.508. Our models yielded high predictive performance and may have the potential to be utilized as both a COVID-19 screening tool and a method to identify health workers at greatest risk of COVID-19 positivity, and therefore most in need of testing.