words)Objective To review and critically appraise published and preprint reports of models that aim to predict either (i) presence of existing COVID-19 infection, (ii) future complications in individuals already diagnosed with COVID-19, or (iii) models to identify individuals at high risk for COVID-19 in the general population.Design Rapid systematic review and critical appraisal of prediction models for diagnosis or prognosis of COVID-19 infection. Data sourcesPubMed, EMBASE via Ovid, Arxiv, medRxiv and bioRxiv until 24 th March 2020. Study selectionStudies that developed or validated a multivariable COVID-19 related prediction model. Two authors independently screened titles, abstracts and full text. Data extractionData from included studies were extracted independently by at least two authors based on the CHARMS checklist, and risk of bias was assessed using PROBAST.Data were extracted on various domains including the participants, predictors, outcomes, data analysis, and prediction model performance.Results 2696 titles were screened. Of these, 27 studies describing 31 prediction models were included for data extraction and critical appraisal. We identified three models to predict hospital admission from pneumonia and other events (as a proxy for covid-19 pneumonia) in the general population; 18 diagnostic models to detect COVID-19 infection in symptomatic individuals (13 of which were machine learning utilising computed tomography (CT) results); and ten prognostic models for predicting mortality risk, progression to a severe state, or length of hospital stay. Only one of these studies used data on COVID-19 cases outside of China.Most reported predictors of presence of COVID-19 in suspected patients included age, body temperature, and signs and symptoms. Most reported predictors of severe prognosis in

Objectives: Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations.Study Design and Setting: We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data.Results: Of 86 identified prediction modeling studies, 52 (60%) presented ROC curves without thresholds and one (1%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds.Conclusion: ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive.

Objective Previous work has suggested that the ultrasound‐based benign simple descriptors (BDs) can reliably exclude malignancy in a large proportion of women presenting with an adnexal mass. This study aimed to validate a modified version of the BDs and to validate a two‐step strategy to estimate the risk of malignancy, in which the modified BDs are followed by the Assessment of Different NEoplasias in the adneXa (ADNEX) model if modified BDs do not apply. Methods This was a retrospective analysis using data from the 2‐year interim analysis of the International Ovarian Tumor Analysis (IOTA) Phase‐5 study, in which consecutive patients with at least one adnexal mass were recruited irrespective of subsequent management (conservative or surgery). The main outcome was classification of tumors as benign or malignant, based on histology or on clinical and ultrasound information during 1 year of follow‐up. Multiple imputation was used when outcome based on follow‐up was uncertain according to predefined criteria. Results A total of 8519 patients were recruited at 36 centers between 2012 and 2015. We excluded patients who were already in follow‐up at recruitment and all patients from 19 centers that did not fulfil our criteria for good‐quality surgical and follow‐up data, leaving 4905 patients across 17 centers for statistical analysis. Overall, 3441 (70%) tumors were benign, 978 (20%) malignant and 486 (10%) uncertain. The modified BDs were applicable in 1798/4905 (37%) tumors, of which 1786 (99.3%) were benign. The two‐step strategy based on ADNEX without CA125 had an area under the receiver‐operating‐characteristics curve (AUC) of 0.94 (95% CI, 0.92–0.96). The risk of malignancy was slightly underestimated, but calibration varied between centers. A sensitivity analysis in which we expanded the definition of uncertain outcome resulted in 1419 (29%) tumors with uncertain outcome and an AUC of the two‐step strategy without CA125 of 0.93 (95% CI, 0.91–0.95). Conclusion A large proportion of adnexal masses can be classified as benign by the modified BDs. For the remaining masses, the ADNEX model can be used to estimate the risk of malignancy. This two‐step strategy is convenient for clinical use. © 2022 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.